- We are seeking a highly skilled and technical Data Developer to join our team. This role will focus on data modeling, building, and optimizing data engineering pipelines, implementing CI/CD practices, and developing serverless functions across AWS and Azure environments. The ideal candidate will be adept at managing both data engineering and infrastructure requirements to support seamless data processing, deployment, and monitoring.
EDUCATION & EXPERIENCE REQUIRED
- Bachelor's degree in Computer Science, Data Development, Information Technology, or a related field.
- 7+ years of experience in data engineering, data architecture, or similar technical roles.
- Demonstrated expertise in database systems, data warehousing, and ETL processes.
- 6+ years as a Python programming language expert.
REQUIRED SKILLS & COMPETENCIES
- ETL/ELT Processes: Building data pipelines using a mix of stored procedures, views, and Python scripts. Knowledge of tools like dbt (for SQL transformations) or simple orchestration (e.g., Airflow basics) is a plus.
- Data Quality & Testing: Implementing checks, validation, and monitoring for pipelines.
- Version Control & Collaboration: Git for code (Python + SQL scripts), plus experience with CI/ CD basics.
- Performance & Scalability: Tuning for large volumes, understanding indexing/partitioning impacts, and basic cloud database concepts.
- Soft Skills: Problem-solving for debugging complex queries/pipelines, clear documentation of models and procedures, and ability to collaborate with members of a team.
- Strong experience in designing relational schemas.
- Entity-Relationship (ER) modeling, dimensional modeling, and understanding trade-offs between normalization vs. denormalization.
- Practical Application: Creating efficient table structures, relationships, constraints, and indexes to support query performance and scalability.
ADVANCED SQL & DATABASE QUERYING
- Mastery of Complex Queries: JOINs (all types), subqueries, CTEs (Common Table Expressions), window functions (ROW_NUMBER, RANK, LAG/LEAD, etc.), aggregation, pivoting, and recursive queries.
- Query Optimization & Performance Tuning: Indexing strategies, execution plans, partitioning, and handling large datasets efficiently.
- Advanced Techniques: CASE statements, dynamic SQL, error handling, and working with timestamps/time zones.
DATABASE ADMINISTRATION & OBJECTS
- Creating and managing stored procedures (for encapsulating business logic, ETL processes, and reusable code).
- Creating and optimizing views (including materialized views where supported) for exposing cleaned/transformed data to reporting applications.
- Other DBA Tasks: Triggers, user-defined functions, permissions/security, backups, and basic maintenance.
- Familiarity with ACID properties, constraints, and database-specific features.
PYTHON PROGRAMMING FOR DATA ENGINEERING
- Solid proficiency in Python for scripting, automation, and data manipulation.
- Key Libraries: pandas (data wrangling), NumPy, SQLAlchemy or psycopg2/pyodbc (for database connectivity), requests (APIs), and basic ETL scripting.
- Writing Modular Code: Functions, classes, error handling, logging, and integrating Python with databases (e.g., running queries, loading data, or orchestrating processes).
- Experience building or supporting data pipelines that combine Python with SQL (e.g., Python calling stored procedures or handling complex transformations).
RELATIONAL DATABASE EXPERIENCE
- Hands-on work with at least one major RDBMS (e.g., SQL Server for T-SQL/stored procedures, PostgreSQL, MySQL Preferably).
- Understanding of database-specific dialects, as stored procedures and advanced features vary (e.g., T-SQL vs. PL/pgSQL).
- Excellent problem-solving and analytical abilities with attention to detail.
- Strong communication and collaboration skills in cross-functional and distributed teams.