Introduction to ADVANA
IVERTIX supports many state-of-the-art Data Platforms for its Government and defense clients using cutting edge technologies available today in the cloud. These data platforms enable the organizations to support their mission critical initiatives leveraging advanced data analytics and machine learning capabilities.
Advana is a multi-domain technology platform which offers military and business decision makers, analysts, and users at all levels unprecedented access to authoritative enterprise data and structured analytics in a scalable, reliable, and secure environment. It offers platform-, data analytics-, and data science-as-a-service tools to complete analysis and data exploration and data visualization capabilities. Using one central data platform with right-time data, data tools, and other self-service products, the goal of Advana is to simplify solutions and put the power of analytics in the pocket of every analyst and decision-making authority at the Department of Defense.
The Mid-Level Data Engineer – Role Description & Requirements
Data is absolutely the lifeblood of Advana and our data engineers are the doctors administering it! Our data engineers support data collection, ingestion, validation, and loading of optimized data in the appropriate data stores. They work on a team made up of analyst(s), developer(s), data scientist(s), and a product lead and everyone on the team collaborates in support of a specific the mission. Working directly with the analyst(s) and the product lead, the data engineer identifies and implements solutions for the data requirements, including building pipelines to collect data from disparate, external sources, implementing rules to validate that expected data is received, cleansed, transformed, massaged and in an optimized output format for the data store. The Data Engineer performs validation and analytics corresponding with client requirements and evolves solutions through automation, optimizing performance with minimal human involvement. As pipelines are executed, the data engineer monitors their status, performance, and troubleshoots issues while working on improvements to ensure the solution is the very best version to address the customer need.
As a Mid-Level Data Engineer, this role focuses specifically on the development and maintenance of scalable data stores that supply big data in forms needed for business analysis. The best athlete candidate for this position will be able to apply advanced consulting skills, extensive technical expertise and has full industry knowledge to develop innovative solutions to complex problems. This candidate is able to work without considerable direction and may mentor or supervise other team members.
What we’re looking for:
- Someone with a solid background developing solutions for high volume, low latency applications and can operate in a fast paced, highly collaborative environment.
- A candidate with distributed computer understanding and experience with SQL, Spark, ETL.
- A person who appreciates the opportunity to be independent, creative and challenged.
- An individual with a curious mind, passionate about solving problems quickly and bringing innovative ideas to the table.
Basic Qualifications:
- 2+ years of experience with Python and SQL
- Experience with connecting to other data systems through APIs, Web Services, SFTP, etc.
- Knowledge of the software development life cycle, testing, and version control
- Experience working with structured, unstructured and/or semi-structured data
- Experience working with data science projects and integrating AI workflows
- Active Secret clearance or Clerable
Additional Qualifications:
- Experience with event-driven micro-service architectures
- Experience with distributed programming languages and paradigms such as Spark, Scala, etc.
- Experience with data engineering tools such as Databricks, ElasticSearch, Apache NiFi, StreamSets, etc.
- Experience with data lakes, data warehouses, and/or data lake houses
- Experience with different database paradigms including but not limited to relational databases (Deltalake, SQL/PostgreSQL/MySQL/etc), document storage databases (NoSQL, MongoDB, etc), Graph databases (Neo4J, etc)
- Experience with cloud services such as AWS S3, Athena, Glue, Lambda, SQS, or Azure Blob, Data Factory, Functions, Storage Queues, etc.
- Experience with data governance and data quality
- Experience with CI/CD, such as Gitlab CI/CD, GitHub Actions, Tekton, Jenkins, Docker, etc.
- Experience with Test Driven Development and Test Frameworks, such as pytest or unittest
- Experience with Data Catalog tools, such as Collibra, Alation, etc.
- Possession of excellent verbal and written communication skills
- Masters degree in related field