Senior Data Engineer

About the Role

Techylla is seeking an experienced Senior Data Engineer to design, build, and optimise enterprise-grade ELT pipelines for our cloud-based data platform. You will be a core contributor to our data infrastructure — ingesting data from multiple operational source systems and transforming it into reliable, governed, and performant datasets for analytics and reporting.

This role is central to our ELT pipeline operations — built primarily on Matillion — and you will also contribute to our Snowflake-based data warehouse initiatives.

If you take pride in writing clean, efficient, testable ELT code and want to help shape how a modern cloud data platform scales, this is the role for you.

Key Responsibilities

ELT Pipeline Development

• Design and develop orchestration and transformation jobs — leveraging Matillion — for batch and near-real-time ingestion across diverse source systems.
• Configure API-based (REST/SOAP), file-based (CSV, JSON, Parquet, XML), and database ingestion pipelines with robust error handling and alerting.
• Build and maintain incremental load and Change Data Capture (CDC) strategies to ensure low-latency, cost-efficient data freshness.
• Implement scheduling, dependency handling, and failure recovery mechanisms — pipelines must be production-grade, not just functional.
• Optimise ELT job performance and minimise cloud data warehouse compute costs through smart pipeline design, query push-down, and efficient scheduling.

Snowflake Platform Engineering

• Contribute to the design and build of Snowflake data warehouse — including database architecture, virtual warehouse sizing, and role-based access control.
• Develop Snowflake-native solutions: Snowpipe for continuous ingestion, Streams and Tasks for CDC and near-real-time processing, and Dynamic Tables where applicable.
• Implement Snowflake clustering keys, materialised views, and result caching strategies to optimise query performance at scale.
• Work with Snowflake's data sharing, data marketplace, and secure view features to support governed data access across teams.
• Manage Snowflake environment configuration including time travel, fail-safe settings, resource monitors, and cost governance controls.

Data Modelling & Platform Architecture

• Implement and maintain the ingestion framework including schema design, staging layers, and data zones (raw → cleansed → curated).
• Apply dimensional modelling, Data Vault 2.0, or medallion architecture patterns as appropriate to each use case, following agreed team standards.
• Manage platform configuration including security roles, data masking, column-level encryption, and access control policies.
• Support PII-compliant ingestion processes in line with GDPR and internal data governance policies.
• Contribute to data quality checks, testing frameworks, and ongoing monitoring of pipeline health and SLA adherence.

Collaboration & Standards

• Partner with business stakeholders, data analysts, and BI developers to translate requirements into scalable, documented data solutions.
• Produce clear technical documentation: pipeline designs, data dictionaries, architecture decision records (ADRs), and operational runbooks.
• Participate in code reviews, enforce development standards, and mentor junior engineers where applicable.
• Contribute to data governance practices including lineage tracking, metadata management, and data quality frameworks.

Must-Have Skills & Qualifications

• 3+ years of hands-on experience in Data Engineering roles delivering production pipelines at scale.
• 1+ years of hands-on ELT tool experience — Matillion strongly preferred — covering orchestration jobs, transformation jobs, component configuration, and scheduling. Production-level, not exploratory.
• 2+ years working with cloud-based data warehouse or lakehouse platforms — Snowflake strongly preferred; Databricks or Redshift also considered.
• Proven experience designing and delivering enterprise ELT pipelines covering API, file, and database sources.
• Strong SQL development skills: query optimisation, window functions, CTEs, execution plan analysis, and performance tuning.
• Solid understanding of incremental loading strategies, watermarking, and CDC patterns.
• Experience with file-based ingestion (CSV, JSON, Parquet, XML) and schema evolution handling.
• Ability to own work end-to-end — from requirement gathering through to documented, production-ready delivery.

Nice to Have

• Snowflake Advanced Features: Hands-on experience with Snowpipe, Streams, Tasks, Dynamic Tables, Snowflake Cortex, or data sharing features.
• Orchestration Tools: Familiarity with Apache Airflow, dbt (data build tool), Prefect, or similar pipeline orchestration and transformation frameworks.
• Data Modelling: Knowledge of Data Vault 2.0, dimensional modelling, or medallion (bronze-silver-gold) architecture patterns.
• Governance & Compliance: Experience with PII handling, GDPR-compliant ingestion, audit logging, and data lineage tooling (e.g., Alation, Collibra, or OpenLineage).
• ESG / Sustainability Data: Prior exposure to ESG reporting platforms or sustainability data workflows — auditability and traceability requirements are a key part of our platform.
• Matillion Certification: Matillion Certified Associate or Professional — or equivalent ELT tool certification.
• Snowflake Certification: SnowPro Core or SnowPro Advanced: Data Engineer certification.
• Version Control & CI/CD: Experience managing ELT pipeline code (e.g. Matillion jobs) in Git and contributing to CI/CD pipelines for data infrastructure.

What We Offer

• A dynamic, high-growth environment.
• Opportunity to work on impactful data projects with cross-functional visibility across the organisation.
• Competitive compensation aligned with market standards.
• Collaborative team culture that values clarity of communication and ownership of outcomes.