REALSCOUT | Senior Data Engineer / Data Architect - Data Pipeline | REMOTE (minimum 5-hour overlap with Pacific US Timezone) | Full-Time
This role is specifically to work on our data pipeline - the core of our technology. We're flexible on title; the only hard requirement is that you're senior in experience. The pipeline is responsible for providing agents, brokers, and homebuyers real estate updates from 100+ nation-wide data feeds as quickly as possible. We’re looking for someone with at-scale experience to make improvements in and to the pipeline’s architecture -- currently Apache Airflow, Golang, Python, AWS, and Postgres, but flexible. Deployment, logging, metrics collection, SLA improvements: everything is fair game!
A typical week will entail:
- Ensuring perfect replication of 100+ real estate data feeds with as little lag as possible
- Scaling a daily emailer from 100k to 1m personalized sends
- Expanding our set of attributes that no one else in the industry has, like "stainless steel appliances" and "near Google shuttle stops"
- Experience with medium-to-large data pipelines: implementing, testing, instrumenting, and deploying
- Experience with stream processing tools such as Kafka, Kinesis, Spark, Storm, and/or Flink
- Familiarity with Python+Go (bonus points for Ruby, which the main website runs)
- Familiarity with automated unit and integration testing
- Experience with a wide variety of data stores such as PostgreSQL, ElasticSearch, and Redshift
- Experience with one major cloud provider (Google, Azure, AWS). AWS a plus.
After you submit an application, if it looks like there's a good fit, we'll reach out to schedule an initial 20 minute conversation for introductions and to answer your questions about RealScout. In the meantime, visit learn.realscout.com/about for more info.