Writing code for over 20 years Ian has worked on everything from API development to data engineering and currently he is a Tech Lead for the Data Intelligence team at Compass. Ian currently does a lot of work with distributed systems, search, data processing and analytics. His current passion project is writing a full text distributed search engine using existing databases like Postgres and MySQL as the data store.
Data processing is a part of many disciplines in technology and as the amount of data you need to process grows so do the techniques you need to be able to safely and speedily ingest that data. Join us as we go through the building blocks Elixir offers to build different ingestion topologies that handle things like backpressure, partitioning, retries, and batching.