Consumer Analytics - Event Collector

1 - About

A collector collects event send by a tracker

The event and data send are describe in a measurement protocol

3 - Aggregation

Data aggregation refers to techniques for gathering individual data records (for example log records) and combining them into a large bundle of data files.

Why aggregation ? Before processing your data, Hadoop splits your data (files) into multiple chunks. After splitting the file(s), a single map task processes each part. If you are using HDFS as the underlying data storage, the HDFS framework has already separated the data files into multiple blocks. In addition, since your data is fragmented, Hadoop uses HDFS data blocks to assign a single map task to each HDFS block.

4 - Protocol

A collector would implement this two HTTP method

4.1 - Post

Post


User-Agent: user_agent_string
POST https://www.example.com/collect
payload_data

4.2 - Get


GET /collect?payload_data HTTP/1.1
Host: https://www.example.com
User-Agent: user_agent_string

A Get request can be triggered:

To avoid to hit a cached HTTP GET requests, a collector should provide a cache burster (ie a special parameter that can be set with a random number in order to ensure that all requests are unique, and that subsequent requests are not retrieved from the cache).

Example from Google Analytics with the z parameter: https://www.example.com/collect?payload_data&z=123456

4.3 - Error handling

If you do not get a 2xx status code, you should NOT retry the request. Instead, you should stop and correct any errors in your HTTP request.

5 - List

6 - Documentation / Reference

  • AWS_Amazon_EMR_Best_Practices.pdf

Data Science
Data Analysis
Statistics
Data Science
Linear Algebra Mathematics
Trigonometry

Powered by ComboStrap