HDFS Namenode

The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself.

Prerequisite

For Virtual Machines, install the Linux Agent.

Configuring the credentials

Configure the credentials in the directory /opt/opsramp/agent/conf/app.d/creds.yaml

hdfs-namenode:
- name: hdfs-namenode
  user: <username>
  pwd: <Password>
  encoding-type: plain
  labels:
    key1: val1
    key2: val2

Configure the application

Virtual machine

Configure the application in the directory /opt/opsramp/agent/conf/app/discovery/auto-detection.yaml

- name: hdfs-namenode
  instance-checks:
    process-check:
      - namenode
    port-check:
      - 50070
      - 9870
    uri-check:
      - "jmx?qry=Hadoop:service=NameNode,name=FSNamesystem*"
      - "jmx?qry=Hadoop:service=NameNode,name=NameNodeInfo"

Docker environment

Configure the application in the directory /opt/opsramp/agent/conf/app/discovery/auto-container-detection.yaml

- name: hdfs-namenode
  container-checks:
    image-check:
      - namenode
    port-check:
      - 50070
      - 9870

Kubernetes environment

Configure the application in config.yaml

- name: hdfs-namenode
  container-checks:
    image-check:
      - namenode
    port-check:
      - 50070
      - 9870

Validate

Go to Resources under the Infrastructure tab to check if your resources are onboarded and the metrics are collected.

Metrics


OpsRamp Metric	Metric Display Name	Unit	Description
hdfs_namenode_capacity_total	Capacity Total	bytes	Total disk capacity
hdfs_namenode_capacity_used	Capacity Used	bytes	Disk usage
hdfs_namenode_capacity_used_percent	Capacity Used Percent	percent	Disk usage
hdfs_namenode_capacity_remaining	Capacity Remaining	bytes	Remaining disk space left
hdfs_namenode_capacity_remaining_percent	Capacity Remaining Percent	percent	Remaining disk space left
hdfs_namenode_total_load	Total Load		Total load on the file system
hdfs_namenode_fs_lock_queue_length	Fs Lock Queue Length		Lock queue length
hdfs_namenode_blocks_total	Blocks Total	Blocks	Total number of blocks
hdfs_namenode_max_objects	Max Objects		Maximum number of files that HDFS supports
hdfs_namenode_files_total	Files Total	Files	Total number of files
hdfs_namenode_pending_replication_blocks	Pending Replication Blocks	Blocks	Number of blocks pending replication
hdfs_namenode_under_replicated_blocks	Under Replicated Blocks	Blocks	Number of under replicated blocks
hdfs_namenode_scheduled_replication_blocks	Scheduled Replication Blocks	Blocks	Number of blocks scheduled for replication
hdfs_namenode_pending_deletion_blocks	Pending Deletion Blocks	Blocks	Number of pending deletion blocks
hdfs_namenode_num_live_data_nodes	Num Live Data Nodes	Nodes	Total number of live data nodes
hdfs_namenode_num_dead_data_nodes	Num Dead Data Nodes	Nodes	Total number of dead data nodes
hdfs_namenode_num_decom_live_data_nodes	Num Decom Live Data Nodes	Nodes	Number of decommissioning live data nodes
hdfs_namenode_num_decom_dead_data_nodes	Num Decom Dead Data Nodes	Nodes	Number of decommissioning dead data nodes
hdfs_namenode_volume_failures_total	Volume Failures Total		Total volume failures
hdfs_namenode_estimated_capacity_lost_total	Estimated Capacity Lost Total	bytes	Estimated capacity lost
hdfs_namenode_num_decommissioning_data_nodes	Num Decommissioning Data Nodes	Nodes	Number of decommissioning data nodes
hdfs_namenode_num_stale_data_nodes	Num Stale Data Nodes	Nodes	Number of stale data nodes
hdfs_namenode_num_stale_storages	Num Stale Storages		Number of stale storages
hdfs_namenode_missing_blocks	Missing Blocks	Blocks	Number of missing blocks
hdfs_namenode_corrupt_blocks	Corrupt Blocks	Blocks	Number of corrupt blocks
hdfs_namenode_num_failed_data_nodes	Num Failed Data Nodes		Total number of failed data nodes
hdfs_namenode_nondfs_used_percent	Nondfs Used Percent	Percent	Total space used by NonDfs