Arth-Task-7(How to share Storage on fly(Elasticity to DataNode)?)

6 min readMar 12, 2021

🌀 7.1: Elasticity Task
Step1:-Integrating LVM with Hadoop and providing Elasticity to DataNode Storage
step2:- Increase or Decrease the Size of Static Partition in Linux.
step3:- Automating LVM Partition using Python-Script.

Solution of the task……………………………….

In Linux, Logical Volume Manager (LVM) is a device mapper framework that provides logical volume management for the Linux kernel. Most modern Linux distributions are LVM-aware to the point of being able to have their root file systems on a logical volume.

Heinz Mauelshagen wrote the original LVM code in 1998, when he was working at Sistina Software, taking its primary design guidelines from the HP-UX’s volume manager.

LVM is used for the following purposes:

Creating single logical volumes of multiple physical volumes or entire hard disks (somewhat similar to RAID 0, but more similar to JBOD), allowing for dynamic volume resizing.
Managing large hard disk farms by allowing disks to be added and replaced without downtime or service disruption, in combination with hot swapping.
On small systems (like a desktop), instead of having to estimate at installation time how big a partition might need to be, LVM allows filesystems to be easily resized as needed.
Performing consistent backups by taking snapshots of the logical volumes.
Encrypting multiple physical partitions with one password.

LVM can be considered as a thin software layer on top of the hard disks and partitions, which creates an abstraction of continuity and ease-of-use for managing hard drive replacement, repartitioning and backup.

Basic functionality

Volume groups (VGs) can be resized online by absorbing new physical volumes (PVs) or ejecting existing ones.
Logical volumes (LVs) can be resized online by concatenating extents onto them or truncating extents from them.
LVs can be moved between PVs.
Creation of read-only snapshots of logical volumes (LVM1), leveraging a copy on write (CoW) feature,or read/write snapshots (LVM2)
VGs can be split or merged in situ as long as no LVs span the split. This can be useful when migrating whole LVs to or from offline storage.
LVM objects can be tagged for administrative convenience.
VGs and LVs can be made active as the underlying devices become available through use of the lvmetad daemon.

Create LVM Partitions using some steps…………………

Step1:- ADD The storage manualy on the Virtual Machine.

Solving this step we use virtual machine and add the storage to the virtual machine, we add two storage size of 20,40.

step2:- Run the Virtual machine and run command to check storage add on systems or not.

In this step we run command like,

fdisk -l

This command use to show Hardisk is add or not.

Step3:- Run command to check to create storage mount or not.

Solving this step we use command like

df -h

step4:- Create physical volumes of new storage.

In this step we create Physical volumes of new storage. Run the command pvdispaly to chek how many physical device add virtual machine, Creating Physical device we use command like pvcreate.

Use pvreate command to create physical device.

Step5:- Create Volume Groups to add Physical volumes .

In this step we create one virtual groups to mount new physical volumes, using command like vgcreate use this command to create virtual groups, showing virtual groups use command like vgdisplay.

vgdisplay

Creating a volume groups we use command like

vgcreate ***namevolume*** ***partionname***

Step6:- Create LVM partitions.

In this step we create final step creating LVM partitions. Using command like lvcreate and use to show partition we use lvdisplay commands.