Partitioning setup for Linux from Scratch in VirtualBox

I’ve finally taken the plunge and committed, to untarring and compiling, a bucket load of source code to complete Linux from Scratch. I’ll be documenting some of my setup here. I’m far from an expert, that’s why I’m doing this, but if you have any constructive criticism I’d be glad to hear it. I’m using VirtualBox and an installation of CentOS to build LFS.

The first task I’ll be undertaking is partitioning a disk ready for my LFS setup. I’ve designed my partition setup based on the advice in the LFS manual: Creating a New Partition.

Partition Size (GB/MB) On Primary
/ (root partition) 10GB 1
/home 10GB 1
/usr 5GB 1
/opt 10GB 1
/swap 2GB 0
/boot 100M 0

* The On Primary = 1 means the partition will be hosted on the first, larger, partition we create.

Add a VirtualBox HDD

First add VirtualBox hard disk. I added a 50GB drive in VB for this to give me plenty of space for my LFS installation.

LFS 300x237 Partitioning setup for Linux from Scratch in VirtualBox

Identify the new device

Boot up the host OS, CentOS 7 in my case, and open a command prompt once logged in. The command lsblk can be used to quickly identify the new device. From the output we can easily identify the disk as sdb. It is 50GB and contain no partitions.

linux> lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   50G  0 disk 
├─sda1            8:1    0  500M  0 part /boot
└─sda2            8:2    0 49.5G  0 part 
  ├─centos-swap 253:0    0    2G  0 lvm  [SWAP]
  └─centos-root 253:1    0 47.5G  0 lvm  /
sdb               8:16   0   50G  0 disk 
sr0              11:0    1 1024M  0 rom  

Create a large Primary Partition

First we will create a single large partition. This partition will be the logical container for the above partition mark above as On primary = 1. We will be using fdisk for this. You’ll need to run fdisk as the root user.

fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xdf241aa7.

Command (m for help): 

Enter ‘n’ to create a new partition.
Enter ‘e’ for partition type.
Enter ‘1’ for partition number.
Accept the default for first sector.
Enter ‘+40G’ for the last sector.

This partition does not yet exist, we have to tell fdisk to writes changes before that happens. However you can have a look at what will be done by entering the command ‘p’…

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    83888127    41943040    5  Extended

Now we will add the four logical partitions inside the one above…

Enter ‘n’ to create a new partition.
Enter ‘l’ for logicial.
Accept the default for first sector.
For last sector enter ‘+10G’

Enter ‘n’ to create a new partition.
Enter ‘l’ for logicial.
Accept the default for first sector.
For last sector enter ‘+10G’

Enter ‘n’ to create a new partition.
Enter ‘l’ for logicial.
Accept the default for first sector.
For last sector enter ‘+5G’

Enter ‘n’ to create a new partition.
Enter ‘l’ for logicial.
Accept the default for first sector.
For last sector enter ‘+10G’

Enter the ‘p’ command to print out the partition table

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    83888127    41943040    5  Extended
/dev/sdb5            4096    20975615    10485760   83  Linux
/dev/sdb6        20977664    41949183    10485760   83  Linux
/dev/sdb7        41951232    52436991     5242880   83  Linux
/dev/sdb8        52439040    73410559    10485760   83  Linux

Finally we will add partitions for the swap and boot partitions. While still in fdisk…

Enter ‘n’ to create a new partition.
Enter ‘p’ for primary.
Accept the default for first sector and partition number.
For last sector enter ‘+2G’

Enter ‘n’ to create a new partition.
Enter ‘p’ for primary.
Accept the default for first sector and partition number.
For last sector enter ‘+100M’

Enter ‘p to print the output. You should have something like below.

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    83888127    41943040    5  Extended
/dev/sdb2        83888128    88082431     2097152   83  Linux
/dev/sdb3        88082432    88287231      102400   83  Linux
/dev/sdb5            4096    20975615    10485760   83  Linux
/dev/sdb6        20977664    41949183    10485760   83  Linux
/dev/sdb7        41951232    52436991     5242880   83  Linux
/dev/sdb8        52439040    73410559    10485760   83  Linux

Finally enter ‘w’ to write the changes to disk and exit. You can use lsblk again to get a more human friendly view of the partition sizes.

NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   50G  0 disk 
├─sda1            8:1    0  500M  0 part /boot
└─sda2            8:2    0 49.5G  0 part 
  ├─centos-swap 253:0    0    2G  0 lvm  [SWAP]
  └─centos-root 253:1    0 47.5G  0 lvm  /
sdb               8:16   0   50G  0 disk 
├─sdb1            8:17   0    1K  0 part 
├─sdb2            8:18   0    2G  0 part 
├─sdb3            8:19   0  100M  0 part 
├─sdb5            8:21   0   10G  0 part 
├─sdb6            8:22   0   10G  0 part 
├─sdb7            8:23   0    5G  0 part 
└─sdb8            8:24   0   10G  0 part 
sr0              11:0    1 1024M  0 rom  

Format the partitions and give them each a table

From the output of lsblk you can match up the devices using their sizes and give them a label.

sudo mkfs -v -t ext4 /dev/sdb7 -L usr 
sudo mkfs -v -t ext4 /dev/sdb3 -L boot
sudo mkfs -v -t ext4 /dev/sdb5 -L root
sudo mkfs -v -t ext4 /dev/sdb6 -L home
sudo mkfs -v -t ext4 /dev/sdb8 -L opt
sudo mkswap /dev/sdb2 -L swap

Mount the partitions

Next create some folders to mount the partitions on….

mkdir /mnt/lfs
chown -R rhys:users /mnt/lfs
export LFS=/mnt/lfs
mount -v -t ext4 /dev/sdb5 $LFS
mkdir /mnt/lfs/usr
mount -v -t ext4 /dev/sdb7 $LFS/usr
mkdir /mnt/lfs/home
mount -v -t ext4 /dev/sdb6 $LFS/home
mkdir /mnt/lfs/opt
mount -v -t ext4 /dev/sdb8 $LFS/opt
mkdir /mnt/lfs/boot
mount -v -t ext4 /dev/sdb3 $LFS/boot
swapon /dev/sdb2

You can view the new mounts with..

linux> df -h
...
dev/sdb5                9.8G   37M  9.2G   1% /mnt/lfs
/dev/sdb7                4.8G   20M  4.6G   1% /mnt/lfs/usr
/dev/sdb6                9.8G   37M  9.2G   1% /mnt/lfs/home
/dev/sdb8                9.8G   37M  9.2G   1% /mnt/lfs/opt
/dev/sdb3                 93M  1.6M   85M   2% /mnt/lfs/boot
...

You may receive a warning about these mounts…

You just mounted an file system that supports labels which does not
contain labels, onto an SELinux box. It is likely that confined
applications will generate AVC messages and not be allowed access to
this file system.  For more details see restorecon(8) and mount(8).

Fix this with…

linux> restorecon -R /mnt

Making the mounts persistent

You can copy the output from /etc/mtab and add an edited version to your /etc/fstab file to make these mounts persistent.

These mounts should probably be owned by the lfs user. I’ll update this section with more detail when I decide precisely what to do.


Highlight text using Grep without filtering text out

Here’s a neat little trick I learned today I thought was worth sharing. Sometimes I want to highlight text in a terminal screen using grep but without filtering other lines out. Here’s how you do it…

mysqlbinlog mysql-bin.000473 | grep --color -E '^|not closed';

The important part is the regexp in grep. The first section matches any line, but doesn’t colour it, the section after the pipe is the text you want to highlight. Output will look something like below.

binlog not closed properly Highlight text using Grep without filtering text out


MariaDB Compound Statements Outside Stored Procedures

It’s always been a small annoyance that the MySQL / MariaDB flavour of SQL wouldn’t allow you to use if else logic or loops outside of a stored procedure or trigger. There were ways around  this but it’s not as nice if you’re coming from TSQL. This is rectified in MariaDB from 10.1.1.

One thing that is worth noting, which perhaps the manual doesn’t make totally clear, is that you should use the DELIMITER statement in your SQL code. This is just the same as when writing stored procedures. You get syntax error if you don’t do this.

DELIMITER |
BEGIN NOT ATOMIC
 
/* SQL CODE HERE
*/
 
END; |
 
DELIMITER ;

Here are a few examples;

UPDATE: Please note that the source code plugin strips the pipe characters from the code after you click “View Code”

WHILE LOOP

DELIMITER |
 
BEGIN NOT ATOMIC
	DECLARE i INTEGER;
	SET i = 1;
 
	WHILE i < 10 DO
 
		SELECT i;
		SELECT SLEEP(i);
		SET i = i + 1;
 
	END WHILE;
 
END; |
 
DELIMITER ;

IF STATEMENT

DELIMITER |
 
BEGIN NOT ATOMIC
 
	IF 1 = 1 THEN
		SELECT 'Hello, World!';
	END IF;
 
END; |
 
DELIMITER ;

CASE STATEMENT

DELIMITER |
 
BEGIN NOT ATOMIC
 
	CASE(@@port)
		WHEN 3306 THEN
		SELECT 'Running on the default port.';
	ELSE
		SELECT 'You are not running on the default port.';
	END CASE;
 
END; |
 
DELIMITER ;

ITERATE LOOP WITH IF ELSE

DELIMITER |
 
BEGIN NOT ATOMIC
 
	DECLARE i INTEGER DEFAULT 0;
 
	my_loop: LOOP
		SET i = i + 1;
		SELECT i * i;
		IF i < 10 THEN 
			ITERATE my_loop;
		ELSE
			LEAVE my_loop;
		END IF;
	END LOOP my_loop;
END; |
 
DELIMITER ;

TokuDB file & table sizes with information_schema

Here’s a few queries using the information_schema.TokuDB_fractal_tree_info to get the on disk size in MB for TokuDB tables.

This first one will sum up the on disk size for tables using the TokuDB engine.

SELECT table_schema, table_name, SUM(ROUND(bt_size_allocated / 1024 / 1024, 2)) AS table_size_mb
FROM information_schema.`TokuDB_fractal_tree_info`
WHERE table_schema = 'database_name'
GROUP BY table_schema, table_name;

To get a breakdown of the files making up a specific table run the following…

SELECT internal_file_name, SUM(ROUND(bt_size_allocated / 1024 / 1024, 2)) AS file_size_mb
FROM information_schema.`TokuDB_fractal_tree_info`
WHERE `table_schema` = 'database_name'
AND `table_name` = 'table_name'
GROUP BY internal_file_name
WITH ROLLUP

The figure might be slightly off the actual on disk size. I’ve never noticed a difference of more than 0.01% so it’s close enough for most purposes.


6 Useful Bash tips I wish I knew from day zero

Here’s a few bash commands tricks I wished I’d been shown when I first picked up the shell. Please share any additional favorites you have.

Repeat the last command with sudo

How often do you type…

yum install long-list packages-devel

Only to be told…

You need to be root to perform this command.

Execute the following to install your packages…

sudo !!

The !! points to the previous command executed in the shell.

Save a readonly file in vi/vim

How many times have you opened a file in vi/vim, made lots of changes, only to be told it’s read-only when you attempt to save? Enter this in command mode to get around this…

:w !sudo tee

There’s a good explanation of this here.

alias

It can be time-consuming to search through your command history. Setup an alias instead.

alias shortcut="cmd -with=1 --lots=2 --of=3 -options=4 | piped -a -b -c"

The cmd with lots of options can now be executed as…

shortcut

Much simpler! Add these to your .bash_profile file to make them available permanently.

Clear your terminal window

I used to use clear for this. But this just shimmies everything upwards. Use..

reset

To actually clear the terminal screen.

Display output to a screen and save to a file

For a long time I was copying and pasting terminal output to save into text files. Then I discovered tee. Do this instead…

ls -lh | tee output.txt

Command-line calculator

For simple calculations on the command-line you can use bc.

echo "2 * 2" | bc

Outputs…

4