DATA ENGINEER

04039

(Competitive)



DISTINGUISHING FEATURES OF THE CLASS


The work involves expanding the data infrastructure, developing Extract, Transform, and Load (ETL) pipelines and open datasets, and connecting analysts and data scientists as part of the Office of Accountability, Performance and Innovation for the City of Syracuse. Under the general direction of the Director of Innovation, employees in this position are responsible for building pipelines for the City’s performance management dashboard, the open data portal, and other data science projects. Does related work as required.


TYPICAL WORK ACTIVITIES

Sets up automated Extract, Transform, and Load (ETL) workflows.

Works closely with other members of the team to support their data needs.

Assists users in interacting with and connecting to databases.

Designs and maintains data warehouse/reporting schemas and migrates pipelines from legacy systems to new data warehouse.

Researches and implements new tools and techniques to develop ETL pipelines.

Analyzes existing code and makes suggestions for optimizing and migrating existing processes.

Writes scripts to automate manual processes, improve existing processes, and troubleshoot as needed.

Writes program instructions in a specific programming language.

Develops reports, prepares data for export, establishes procedures for importing of data

Analyzes communication requirements with software.

Develops and implements ongoing needs assessment to identify types of contents of training.

Maintains data connectivity security.

Prepares data for display on media such as Internet, Intranet, PDSs, laptops, etc.

Works closely with data analysts and other staff to build data integrations.

Consults with user to ascertain required project scopes and results to solve reporting needs.

Performs cost benefit analysis and feasibility on computer applications.

Devises/applies plans to upgrade from annual methods to computerized systems.

Prepares workflow processes.

Consults with vendors to ascertain the products available to meet the customer’s needs.


FULL PERFORMANCE KNOWLEDGES, SKILLS, ABILITIES AND PERSONAL CHARACTERISTICS


Good knowledge of data engineering principles and process.

Strong analytical, problem solving, and communication skills, including mapping of business needs or issues to technical solutions.

Ability to communicate technical nuances in plain language with partners of varying technical background.

Ability to solve problems and demonstrate critical thinking skills.

Ability to debug software and hardware issues.

Good knowledge of SQL query and data transformation skills.

Knowledge of scripting languages such as Python or Bash, version control and Linux operating systems.

Knowledge of cloud computing ecosystem.

Ability to manage and support automated ETL and data synchronization.

Ability to use good judgement and focus on detail as required.


MINIMUM QUALIFICATIONS

  1. Graduation from a regionally accredited college or university or one accredited by the New York State Board of Regents to grant degrees with a Bachelor’s degree in Computer Science, Mathematics, Engineering, Information Management or a closely related field; OR,

  2. Four (4) years of work experience, or its part-time equivalent, using datasets to manage and process datasets for reporting and analytics as part of a large organization.

10/2019 Date of Original Composition