Hive - Live Long And Process (LLAP)

About

Hive LLAP:

is slider service (long-lived daemon)
supports in-memory data caching
was Introduce in Hive 2.0

Functionality such as caching, pre-fetching, some query processing and access control are moved into the daemon. Small/short queries are largely processed by this daemon directly, while any heavy lifting will be performed in standard YARN containers.

Code: llap-server

Articles Related

Management

Configuration

Conf file. See ConfigurationProperties-LLAP
XML Configuration from llap-daemon-site

curl localhost:15002/conf

Configurations for LLAP monitoring are stored in resources.json, appConfig.json, metainfo.xml which are embedded into templates.py used by Slider.

Status

With web service

curl localhost:15002/status

{
  "status" : "STARTED",
  "uptime" : 139093,
  "build" : "2.1.0-SNAPSHOT from 77474581df4016e3899a986e079513087a945674 by gopal source checksum a9caa5faad5906d5139c33619f1368bb"
}

With client

hive --service llapstatus --name {llap_app_name} [-f] [-w] [-i] [-t]

Jmx

With web service JSON JMX data

curl localhost:15002/jmx

The LLAP Metrics Collection Server collects JMX metrics from all LLAP Daemons periodically. LLAP Monitor Daemon runs on YARN container, similar to LLAP Daemon, and listens on the same port.

Stacks

With web service JVM Stack Traces of all threads -

curl localhost:15002/stacks

Peers (Daemon /)

The list of LLAP Daemons are extracted from the Zookeeper server which launched in the cluster.

With web service

curl localhost:15002/peers

{
  "dynamic" : true,
  "identity" : "718264f1-722e-40f1-8265-ac25587bf336",
  "peers" : [
 {
    "identity" : "940d6838-4dd7-4e85-95cc-5a6a2c537c04",
    "host" : "sandbox121.hortonworks.com",
    "management-port" : 15004,
    "rpc-port" : 15001,
    "shuffle-port" : 15551,
    "resource" : {
      "vcores" : 24,
      "memory" : 128000
    },
    "host" : "sandbox121.hortonworks.com"
  },
]
}