Created at: January 15, 2025 00:17
Company: Accenture
Location: Arlington, VA, 22201
Job Description:
Accenture Flex offers you the flexibility of local fixed-duration project-based work powered by Accenture, a leading global professional services company. Accenture is consistently recognized on FORTUNE's 100 Best Companies to Work For and Diversity Inc's Top 50 Companies For Diversity lists.
As an Accenture Flex employee, you will apply your skills and experience to help drive business transformation for leading organizations and communities. In addition to delivering innovative solutions for Accenture's clients, you will work with a highly skilled, diverse network of people across Accenture businesses who are using the latest emerging technologies to address today's biggest business challenges.
You will receive competitive rewards and access to benefits programs and world-class learning resources. Accenture Flex employees work in their local metro area onsite at the project, significantly reducing and/or eliminating the demands to travel.
We are looking for a Data Engineer to join the team to develop pipelines to migrate and transform data from the Client’s on-premise and other data sources to Azure Data Lake. The role will work with Data Analysts to gather requirements, work with Client teams to setup the interface, design & develop data pipelines using good design practices, including different Ingestion patterns to ingest data into the data lake.
Job Description:
Create data driven generic framework components using Databricks, PySpark and the other data engineering tools
Create new data pipelines leveraging existing data ingestion frameworks, tools
Orchestrate data pipelines using the Azure Data Factory service.
Develop/Enhance data transformations based on the requirements to parse, transform and load data into Enterprise Data Lake, Delta Lake
Perform UnitTesting, coordinate integration testing and UAT
Create HLD/DD/runbooks for the data pipelines
Analyze source data, define DQ Rules to monitor and improve Data Quality
Performance tuning/optimization