Talk 1:
As a large multi partner loyalty program, PAYBACK receives millions of events per day. This talk is about how PAYBACK has implemented a performant and scalable system with Kafka and Spark Streaming for processing these events in realtime. Furthermore we will take a look at how this system is run in production:
- Monitoring the entire system by aggregating log messages via Kafka
- Duplicating data from other data sources within the company (e.g. Oracle databases)
- Automated testing for continuous deployment
Talk 2:
Apache Drill makes the promise to provide a flexible, distributed SQL engine for large number of data sources for interactive data exploration. Supported data sources include Hadoop, NoSQL databases and cloud storage. Compliance with ANSI SQL 2003 ensures fast onboarding of data analysts and industry-standard interfaces like JDBC/ODBC ensure integration with wide number of frontends.
In the talk architecture of Apache Drill will be introduced first. In the second part a potential use case of joining data from a relational database with data on Hadoop and a NoSQL database will be presented.
Let us discuss if Apache Drill satisfies its ambitions getting a Swiss army knife equivalence of a query engine.
maps: Theresienhöhe 12, 80339 München
