Adding Elasticity to the datanode in Hadoop

So what is LVM?

LVM is a tool for logical volume management. With LVM, a hard drive or set of hard drives can be allocated to one or more physical volumes. LVM physical volumes can be placed on other block devices which might span two or more disks.

  1. Next we would be creating the volume groups. Volume groups are like a canvas on which various physical volumes can be put for storage. Volume groups dont have any storage of their own.
  2. Next we will use the volume group as a traditional storage unit . We will partition it and format it like we do always.
  3. We would then mount this partition to the datanode folder.
  4. Then to test the power of LVM we would be expanding the partition on the fly.

Creating physical volume

The normal storage units are not compatible to be used in a volume group. We can see the current attached volumes using the given command.

Creating Volume Group

The next step is to create a volume group. A volume group takes in number of physical volumes and creates a single unit from it

Creating a partition

Now we have got ourselves a storage unit. So the next task would be to create a partition from it. This is called a logical partition. To create a logical partition we use the following command

Increasing the storage

Now I would like to show the power of the LVM storage unit we just created. Let’s say in a certain scenerio we get an urgent need to increase the volume size. Under normal circumstances where we dont have elasticity there would be an increased workload but LVM makes this very easy for us. We just take some storage from the volume group and add it to the system. Let’s see how this is done.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store