Вакансия: Middle/Senior Data Engineer
Компания: Norsec Delta Projects
Локация: Atyrau, Remote 5/2
Зарплатная вилка: 600к+
Требования:
Role Requirement & Responsibilities:
- Design, develop, and maintain relational databases for data storage and data mining.
- Perform data warehouse design and testing, including data design, database architecture, metadata, and repository creation.
- Design data warehouse schematics, layouts, and capacity planning.
- Create and implement effective metrics and monitoring processes.
- Develop processes and procedures for entering information into data warehousing systems and for ensuring reliability of information entered.
- Seek continuous improvement in performance of data warehouse and ensure security of data.
- Considered highly skilled and proficient in discipline.
- Conducts complex work important to the organization.
- Works under minimal supervision with wide latitude for independent judgment.
- Typically requires six to nine years of relevant experience or equivalent combination of experience and education.
- Data extraction from different sources such as SQL Server, flat files, Excel, and XML files into staging, transformation necessary to recast source data for analysis, and loading of transformed data into the presentation repository. It includes the selection criteria to extract data from source systems, perform any necessary data transformations or derivations needed, data quality audits, and cleansing.
- Data Modeling: Design and implementation of data models for both integration and presentation repositories.
- Production Automation: Design and schedule the ETL/ELT process for periodic automated updates, including FTP/SFTP file transfers. Automated update processes must have built-in fail-over strategies and must ensure data consistency and correctness.
- T-SQL Programming: Writing complex stored procedures, efficient SQL Queries with joins, and tuning for optimal performance.
Core Skills:
RDBMS, e.g. Microsoft SQL Server or MySQL
Structured Query Language (SQL), T-SQL
Data Warehousing and ETL and Analytics Tools, e.g. MS tools(SSIS,SSAS,SSRS)
Skills required Listing:
Programming languages VB/C#/Python
Azure DevOps/Agile/Scrum
Basic knowledge of Azure Data Lake Storage, Gen 2
Basic knowledge of Azure Data Factory (and Azure Data Pipelines) for data transformation and orchestration
Basic knowledge of Azure Databricks with an emphasis on data transformation and data cleansing
Basic knowledge of Azure SQL Data Warehouse / Azure Synapse
Desired skills in new cloud tools DeltaLake, Spark SQL, Polybase, Scala
English - Required
Russian - Nice to have
Kazakh - Nice to have
[email protected]
@akmaralomir
Компания: Norsec Delta Projects
Локация: Atyrau, Remote 5/2
Зарплатная вилка: 600к+
Требования:
Role Requirement & Responsibilities:
- Design, develop, and maintain relational databases for data storage and data mining.
- Perform data warehouse design and testing, including data design, database architecture, metadata, and repository creation.
- Design data warehouse schematics, layouts, and capacity planning.
- Create and implement effective metrics and monitoring processes.
- Develop processes and procedures for entering information into data warehousing systems and for ensuring reliability of information entered.
- Seek continuous improvement in performance of data warehouse and ensure security of data.
- Considered highly skilled and proficient in discipline.
- Conducts complex work important to the organization.
- Works under minimal supervision with wide latitude for independent judgment.
- Typically requires six to nine years of relevant experience or equivalent combination of experience and education.
- Data extraction from different sources such as SQL Server, flat files, Excel, and XML files into staging, transformation necessary to recast source data for analysis, and loading of transformed data into the presentation repository. It includes the selection criteria to extract data from source systems, perform any necessary data transformations or derivations needed, data quality audits, and cleansing.
- Data Modeling: Design and implementation of data models for both integration and presentation repositories.
- Production Automation: Design and schedule the ETL/ELT process for periodic automated updates, including FTP/SFTP file transfers. Automated update processes must have built-in fail-over strategies and must ensure data consistency and correctness.
- T-SQL Programming: Writing complex stored procedures, efficient SQL Queries with joins, and tuning for optimal performance.
Core Skills:
RDBMS, e.g. Microsoft SQL Server or MySQL
Structured Query Language (SQL), T-SQL
Data Warehousing and ETL and Analytics Tools, e.g. MS tools(SSIS,SSAS,SSRS)
Skills required Listing:
Programming languages VB/C#/Python
Azure DevOps/Agile/Scrum
Basic knowledge of Azure Data Lake Storage, Gen 2
Basic knowledge of Azure Data Factory (and Azure Data Pipelines) for data transformation and orchestration
Basic knowledge of Azure Databricks with an emphasis on data transformation and data cleansing
Basic knowledge of Azure SQL Data Warehouse / Azure Synapse
Desired skills in new cloud tools DeltaLake, Spark SQL, Polybase, Scala
English - Required
Russian - Nice to have
Kazakh - Nice to have
[email protected]
@akmaralomir