Responsibilities:
• Design, build, and maintain scalable data pipelines on AWS using services such as Amazon S3, Glue, and EMR.
• Develop and optimize ETL processes to ingest data from various sources into our data lake and data warehouse.
• Collaborate with data scientists and analysts to understand data requirements and implement solutions for data transformation, aggregation, and analysis.
• Ensure data quality and integrity by implementing data validation and monitoring processes.
• Optimize performance of data infrastructure and pipelines for efficiency and cost-effectiveness.
• Implement security best practices to protect sensitive data and ensure compliance with regulations such as GDPR and HIPAA.
• Document design decisions, workflows, and operational procedures to maintain a knowledge base for the team.
• Stay up to date with the latest AWS technologies and industry trends to continuously improve our data infrastructure Skills
• Proven experience (3+ years) as a Data Engineer, with a focus on AWS cloud technologies, and experience and knowledge of Microsoft Azure technology is a plus.
• Extensive hands-on experience with AWS services such as Amazon Redshift, Amazon EMR, Amazon Kinesis, AWS Glue, etc Strong understanding of data modeling, ETL processes, data warehousing, and data governance principles.
• Familiarity with programming/scripting languages like Python, and SQL, and experience with data manipulation and transformation.
• Excellent problem-solving skills with the ability to design and implement creative solutions for complex data challenges.
• Knowledge of data governance and compliance standards.
• Strong communication skills to effectively collaborate with technical and non-technical teams.
• Experience in working with Agile methodologies and version control systems.
• Proven ability to work independently and manage multiple priorities effectively.
Bachelors or associate degree in IT discipline Certification in AWS Certified: Solution Architect Associate or AWS Certified: Data Engineer is added advantage. What we need to see from you
• Report to practice lead.
• Work closely with sales team to develop opportunities and solution proposal
• Work within project team to develop and test data analytics platform and model
• Design, develop, and unit test in accordance with established standards.
• Assist project team in analyzing and resolving technical issues.
• Assist in preparing manuals, specification and other technical documentations.
• Package and support deployment releases.
• Assisting in the collection and documentation of user’s requirements, development of user stories, and estimates
Let's Collaborate.
I'd love to connect with you to talk about your project and possible partnership
Full stack mobile (iOS, Android) and web app design and development agency
