Senior Big Data Engineer in Softserve

FULL_TIME

Santiago (hybrid) | Senior | Full time | Programming

Gross Salary $5000 - 5500 USD/month

5 applications
Replies between 3 and 9 days
Last checked 4 days ago
Apply now Quick apply
Requires applying in English

WE ARE
SoftServe is a 12000+ team across the world. At the moment we are growing so you can join one of several dozen projects we are developing right now. Our big, distributed team is working with a fantastic customer, a giant in the mining industry, located in Canada. Our customer is a diversified resource company committed to responsible mining and mineral development with business units focused on steelmaking coal, copper, zinc, and energy. Headquartered in Vancouver, British Columbia (B.C.), Canada, they own or have interests in 13 operating mines, a large metallurgical complex, and several major development projects in the Americas. The client has expertise across a wide range of activities related to exploration, development, mining, and minerals processing, including smelting and refining, health and safety, environmental protection, materials stewardship, recycling, and research.

YOU WANT TO WORK WITH

  • Modern technologies popular on the market
  • Interesting customers
  • Either alone on a subproject or in a team
  • Both developing new functionality and improving the existing ones
  • Practices and approaches to continuous improvement, continuous integration, and delivery processes

YOU ARE

Experienced with Python min 3 years:

  • pyspark
  • pandas
  • airflow

Experienced with the following technologies/frameworks:

  • Azure Data Factory
  • Databricks or HDInsight or Synapse Analytics
  • Azure SQL DW (a.k.a. Synapse SQL Dedicated Pool)
  • Azure Blob Storage and Azure Data Lake
  • Azure SQL DB/Azure SQL MI/SQL Server
  • Azure Functions
  • Apache Spark
  • Event Hubs or similar (Kafka)
  • Azure Stream Analytics or similar (Spark, Flink, Storm)
  • Azure Cosmos DB (nice to have)

In addition:

  • Experience in complex cloud services management concept
  • Networking (virtual networks, firewalls, communication between services).
  • Authentication and security (IAM, service principals and managed identities)
  • Event-driven architecture (Event Grid, Azure Functions, Logic Apps)
  • Hands-on experience with Hadoop ecosystem (e.g. Hive, Impala, Kafka, Spark, HBase, Presto)
  • Hands-on experience with streaming platforms and stream ingestion services (Event Hubs, Kafka).7
  • Hands-on experience with stream analytics concepts and technologies (Flink, Spark, Storm, Azure Stream Analytics)
  • Database modelling, DWH design and implementation experience.
  • ETL/ELT process design and implementation experience.
  • Experienced in massive-parallel-processing (MPP) systems: Azure SQL DW (Synapse), GCP BigQuery, Impala, AWS Redshift
  • Hands-on experience with NoSQL databases
  • Experience in optimization of code, ETL processes and SQL queries.
  • Deep knowledge of SQL language and database management techniques (row-level security, built-in database roles, partitioning, index maintenance)
  • Experience with OLAP systems and reporting tools (Azure Analysis Service, Power BI
  • Git and gitflow
  • Experience with CI/CD process building tools (Azure DevOps,)
  • Project management methodologies (Agile, Waterfall)
  • Strong understanding of security concepts and leading the implementation of practices and approaches in protecting data, systems, and assets in the cloud
  • Accustomed to working in Agile/Scrum team setup (preferred)

TOGETHER WE WILL

  • Be part of challenging projects
  • Be able to work with a diverse team of professionals with a great sense of humor on a challenging project and interesting customer

Flexible hours Flexible schedule and freedom for attending family needs or personal errands.
Internal talks Softserve offers space for internal talks or presentations during working hours.
Meals provided Softserve provides free lunch and/or other kinds of meals.
Paid sick days Sick leave is compensated (limits might apply).
Partially remote You can work from your home some days a week.
Health coverage Softserve pays or copays health insurance for employees.
Fitness subsidies Softserve offers stipends for sports or fitness programs.
Informal dress code No dress code is enforced.
Beverages and snacks Softserve offers beverages and snacks for free consumption.
Vacation on birthday Your birthday counts as an extra day of vacation.

Remote work policy

Hybrid

This job takes place some days from home and others at the office.

About Softserve

We are advisors, engineers, and designers solving business challenges with innovative technology solutions. #StandWithUkraine — Softserve's full profile