Senior Data Engineer
Biobot Analytics
Essential Duties and Responsibilities (What you will be doing):
- Design, build, and maintain robust and scalable data pipelines to collect, process, and store data from various sources (S3, APIs, databases, etc.)
- Develop and optimize data models to support the needs of external product initiatives, as well as requests from internal stakeholders (ex. data scientists, analysts, finance, etc.)
- Create and maintain documentation for data pipelines and data models
- Implement data quality checks, validation processes, and monitoring to maintain high data integrity
- Identify and resolve performance bottlenecks in data pipelines and databases for efficient data processing
- Provide guidance and mentorship to junior data engineers by promoting coding and development best practices (testing, code reviews, etc.)
Education and/or Work Experience Requirements (What you need to succeed):
- BS in Computer Science, Data Analytics, or related, plus 5 years of industry experience, or equivalent
- Expert in Snowflake, or related warehousing framework(s)
- Expert understanding of dimensional data modeling
- Expert in writing SQL, including use of stored procedures, etc
- Expert in one or more programming languages (Python, Scala, etc.)
- Expert in ETL, or related, processes to move and transform data between platforms
- Experience with cloud compute providers (AWS, GCP, Azure, etc.)
- Experience analyzing and optimizing query runtimes
- Experience with CI/CD and common development workflows (Git, code reviews, etc.) for building data models
- Experience with writing tests for, and validating, data models
Education and/or Work Experience Bonuses (What will help you succeed):
- Experience working in a startup environment
- Experience with dbt
- Experience with python
- Experience working with dimensional data models
- Experience working with scientific, geographic, demographic and/or wastewater data
Physical Requirements and Special Demands:
- Required standing or sitting for several hours straight at a time.
- Must be able to talk, listen and speak clearly on telephone and video conferencing for several hours each day.