(Web|Mobile|App) - Analytics (Clickstreams)
About
Web analytics is a event-data application based on event send via HTTP request (hence the web name).
The events are created by a tracker application for events such as click, page load, performance, …
Process
- Tracker - fire events
- Collectors receive events from trackers.
- Apache Kafka
- Amazon Kinesis
- Apache Flume
- Facebook’s Scribe
- Web Service Endpoint with a web Server (Akka Http for Scala (play framework is built on int)),…)
- Enrich:
- Batch: Hadoop Spark
- Stream: Kinesis/Kafka
- Storage:
- Flat file structure on HDFS (S3,…)
- Database (Redshift, Postgres,…)
Platform
https://trends.builtwith.com/analytics
List:
- hotJar
- Facebook Domain Insights
CDP (Customer data pipeline)
Customer data pipeline platform collects, routes, and processes data from source to destination (website, apps, data warehouse, and cloud apps).
- Segment (collect data and hooking up new tools)
- https://www.rudderstack.com/ (1 million monthly events free)
- https://www.mparticle.com/ (60-day free trial)
Open Source
Architecture
- A Content Delivery Network to serve static asset (such as JavaScript)
- A collector endpoint to receives event data (aka pixel data)