senior data engineer

Oversee the requirements, scope, and technical design of integration workflows to ensure
accurate and specification-compliant builds. Additionally, develop and maintain
comprehensive requirements, design documentation, and test plans.
• Seek out, design, and implement internal process improvements: automating
manual processes, optimizing data delivery, re-designing infrastructure for greater
scalability.
• Coordinate with BI Engineers and development teams around data management
including schemas, failure conditions, reconciliation, test data set up, etc.
• Build the infrastructure required for optimal ETL/ELT pipelines to ingest data from a
wide variety of data sources using Microsoft Azure technologies such as Azure Data
Factory and Databricks.
• Construct and maintain of enterprise level integrations using the Snowflake
platform, Azure Synapse, Azure SQL, SSIS and SQL Server.
• Create data tools for data analytics and data science team members that assist
them in building and optimizing our product into an innovative industry leader.
• Design analytics tools that utilize the data pipeline to deliver actionable insights into
customer acquisition, operational efficiency and other key business performance
metrics.
• Troubleshoot issues helping to drive root-cause analysis, and work with
infrastructure teams to resolve incidents and arrive at a permanent resolution.
• Partner with data and analytics teams to strive for greater functionality in our data
systems.
• Provide direction and coordination for development, and support teams, including
globally located resources.
• Understand the layout and working of existing integrations that send and receive
data between Salesforce, Oracle, Concur, JDE, Corporate Data Platform and other
systems.
• Ensure compliance with IT Policy, Sarbanes-Oxley and SOC2 requirements.
• Regularly provides coaching, guidance, and training to less-experienced
analysts/programmers
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:
Required Skills & Experience
• A relevant technical BS Degree in Information Technology
• 5+ years writing SQL queries against any RDBMS with query optimization
• 5 years of data engineering experience leveraging technologies such as Snowflake,
Azure Data Factory, ADLS Gen 2, Logic Apps, Azure Functions, Databricks,
Synapse, SQL Server
• Experience with scripting tools such as Power Shell, Python and XML
• Understanding the pros and cons, and best practices of implementing Data Lake,
using Microsoft Azure Data Lake Storage
• Experience structuring Data Lake for reliability, security and performance
• Experience implementing ETL for Data Warehouse and Business intelligence
solutions
• Skills to read and write effective, modular, dynamic, parameterized and robust code,
establish and follow already established code standards, and ETL framework
• Strong analytical, problem solving, and troubleshooting abilities
• Good understanding of unit testing, software change management, and software
release management
• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as Code
fundamentals
• Experience performing root cause analysis on data and processes to answer
specific business questions, and identify opportunities for improvement
• Experience working within an agile team
• Excellent communication skills
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.