Sr Data Engineer

Required:

●Working experience with Redshift and know the best practices for tuning Redshift’s performance

●Advanced knowledge of Python

●Experience writing and understanding complex SQL queries

●Advanced knowledge of AWS Big Data services, such as EMR, Glue, Athena

●Working experience orchestrating data pipelines withtools like Airflow

●Experience with stream-processing systems such as Apache Spark Streaming, Kinesis, Apache Kafka

●Working experience with RDS, Elasticsearch, Lambda functions, EC2, S3 Working knowledge of messaging and data pipeline tools like Apache Kafka, Kinesis, SNS, SQS

●Working experience with logging and monitoring tools like Elasticsearch Service, Cloudwatch Knowledge of infrastructure as code and CloudFormation

●Working experience on automating the deployment and operation of data pipelines

●Industry Experience: 

○ Involved in developing at least one data pipeline that involved collecting/streaming, storing and processing (ETL) the data for various business use cases.

○Experience with structured, semi-structured and unstructured large data sets from real time/batch streaming data feeds.

Personal Traits Required:

●Experience working in a product company or has product company type software development experience

●Comfortable working in a small company, start-up, fast moving, ambitious environment

●Comfortable working in an environment with ambitious expectations for themselves and each other

●Flexible mindset and able to deal with ambiguity

●Excellent communication - in person, on the phone,and in writing

●Comfortable communicating with a wide range of individuals- including peers, juniors,and seniors and executives

●Intellectual curious, forward thinking, willing to suggest / try new technologies and creative approaches to problems