Archlinux whit RAID and LVM

posted by onThursday February 17th, 2011

This guide explains how to install Archlinux creating a Raid (1) and LVM.
The guide can be general, because the steps to create a software raid and LVM management is the same on every Linux system.

The RAID (Redundant Array of Independent Disks) is defined by wikipedia as: technology that provides increased storage functions and reliability through redundancy.
Definition that I can share.

The use of a RAID system allows for some margin of safety with respect to a disk-failure. For more info about the RAID you can read the information on wikipedia.
In this guide we will use a RAID 1, but can easily be extended to a RAID 5.
For a RAID 1 configuration requires at least two discs, while in a raid 5 will need at least 3.
Our machine has 2 SATA disks (sda and sdb) and a ide cd.
We start the machine with the installation CD (I chose the core version).
AT the login prompt type root as user and then press RETER (the password is not set).
The first step is to set the keyboard in italian (but if you’re reading here maybe you do not need)

[root@archiso ~]# loadkeys it
Loading /usr/share/kbd/keymaps/i386/qwerty/it.map.gz

If you want you can continue setting the network and enable ssh to run the installation remotely (as in this case).

[root@archiso ~]# ifconfig eth0 172.17.13.116 netmask 255.255.255.224
[root@archiso ~]# route add default gw 172.17.13.123
[root@archiso ~]# /etc/rc.d/sshd start

If you want to reach external sites will need to set the dns on /etc/resolv.conf file. Another task will be to provide a password to root user using the passwd command or it will not log in via ssh.

After the login we go to disk partitioning.
The partition layout will be the following:

  • 100MB partition (sda1) to the bootloader which will then be mounted at / boot (RAID 1)
  • 1GB swap partition (sda2 raid 1)
  • the rest of the disc (sda3) for the system and the user’s home (also in a raid 1 but can also be done in raid 5)

NB
The boot partition can not be created with a raid 5 because the boot loader (grub) is not able to access this type of partitions.

The partitioning is made by cfdisck:

[root@archiso ~]# cfdisk /dev/sda

From the screen, select NEW, Primary, set the size to 100 (MB), at the start of free space ( Beginning ),andthen apply the flag for the boot ( bootable ). The partition type will be FD (Linux raid autodetec).
Proceed as above for the swap, taking care to select the new file size and NOT setting the boot flag, the type will always FD.
The rest of the disk space will be used for actual data (root and home) always FD.
After partitioning, you save the partition table by selecting the WRITE command.
These operations are performed on all hard drives you have available.
To speed things up you can make a copy of the partition table of the first disk and then copy it onto others.
To copy, type

[root@archiso ~]# sfdisk -d /dev/sda > table

To set to the new disk

[root@archiso ~]# sfdisk /dev/sdb < table
		Checking that no-one is using this disk right now ...
		OK

		Disk /dev/sdb: 1566 cylinders, 255 heads, 63 sectors/track

		sfdisk: ERROR: sector 0 does not have an msdos signature
		/dev/sdb: unrecognized partition table type
		Old situation:
		No partitions found
		New situation:
		Units = sectors of 512 bytes, counting from 0

		Device Boot    Start       End   #sectors  Id  System
		/dev/sdb1   *        63    192779     192717  fd  Linux raid autodetect
		/dev/sdb2        192780   2184839    1992060  fd  Linux raid autodetect
		/dev/sdb3       2184840  25157789   22972950  fd  Linux raid autodetect
		/dev/sdb4             0         -          0   0  Empty
		Successfully wrote the new partition table

		Re-reading the partition table ...

		If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
		to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
		(See fdisk(8).)

Check with cfdisk that everything went well

[root@archiso ~]# cfdisk /dev/sdb

Now you can move on to creating the raid. The first thing to do is to load the modules to use the raid, will be used only for the module raid1, raid5 in this case is unnecessary

[root@archiso ~]# modprobe raid1
[root@archiso ~]# modprobe raid5

Now you can actually create the raid, to do so using the mdadm utility, please refer to its manual for further information.
To create the first array for the boot partition, type:

[root@archiso ~]# mdadm --create /dev/md1 --level=1 --raid-devices=2 --metadata=0.90 /dev/sda1 /dev/sdb1
		mdadm: array /dev/md1 started.

The –metadata=0.90 parameter is to solve a problem that otherwise would install grub to the partition returned error “Filesystem type unknown, partition type 0xfd“.

Now we create the array for swap and then the data:

	[root@archiso ~]# mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2
		mdadm: Note: this array has metadata at the start and
			may not be suitable as a boot device.  If you plan to
			store '/boot' on this device please ensure that
			your boot-loader understands md/v1.x metadata, or use
			--metadata=0.90
		Continue creating array? y
		mdadm: Defaulting to version 1.2 metadata
		mdadm: array /dev/md2 started.
	[root@archiso ~]# mdadm --create /dev/md3 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3
		mdadm: Note: this array has metadata at the start and
			may not be suitable as a boot device.  If you plan to
			store '/boot' on this device please ensure that
			your boot-loader understands md/v1.x metadata, or use
			--metadata=0.90
		Continue creating array? y
		mdadm: Defaulting to version 1.2 metadata
		mdadm: array /dev/md3 started.

In the event that the partition of the data you wanted to use the raid 5, the command would be (at least 3 disks):

mdadm --create /dev/md3 --level=5 --raid-devices=3 /dev/sda3 /dev/sdb3 /dev/sdc3

and the device / dev/sdcX added to every line of the above.

Now you can check the status of the raid by

[root@archiso ~]# cat /proc/mdstat
		Personalities : [raid1] [raid6] [raid5] [raid4]
		md3 : active raid1 sdb3[1] sda3[0]
			11485379 blocks super 1.2 [2/2] [UU]
			[======>..............]  resync = 30.8% (3541248/11485379) finish=2.6min speed=49579K/sec

		md2 : active raid1 sdb2[1] sda2[0]
			996018 blocks super 1.2 [2/2] [UU]

		md1 : active raid1 sdb1[1] sda1[0]
			96256 blocks [2/2] [UU]

		unused devices: 

To continue you must not wait for the synchronization of the array.
After the creation of the raid can go to the LVM configuration.
To know what LVM is and how it works can be read http://sourceware.org/lvm2/.
The first step is to load the module

[root@archiso ~]# modprobe dm-mod

Now it is time to create the pv (Physical Volume)

[root@archiso ~]# pvcreate /dev/md3
Physical volume "/dev/md3" successfully created

and can be controlled by

[root@archiso ~]# pvdisplay
"/dev/md3" is a new physical volume of "10.95 GiB"
--- NEW Physical volume ---
PV Name /dev/md3
VG Name
PV Size 10.95 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID yMtr3S-0Tl0-Vbuy-dEhV-jx98-1ucX-uts1iY

Now let’s create the vg (Volume Group) dataarray

[root@archiso ~]# vgcreate dataarray /dev/md3
Volume group "dataarray" successfully created

To verify it:

[root@archiso ~]# vgdisplay
--- Volume group ---
VG Name dataarray
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 10.95 GiB
PE Size 4.00 MiB
Total PE 2804
Alloc PE / Size 0 / 0
Free PE / Size 2804 / 10.95 GiB
VG UUID xR51Xu-i34F-hcoP-N1Wp-S1Cm-j2AS-EGghoN

What remains now is to create the lv (Logical Volume) that will become the volumes on which you mount the / and / home

[root@archiso ~]# lvcreate --size 7.9G --name root dataarray
Rounding up size to full physical extent 7.90 GiB
Logical volume "root" created
[root@archiso ~]# lvcreate --size 3.0G --name home dataarray
Logical volume "home" created
[root@archiso ~]# lvdisplay
--- Logical volume ---
LV Name /dev/dataarray/root
VG Name dataarray
LV UUID YRXHC0-L4yv-4Nnk-d6kS-SPEo-Q6kK-xbuz78
LV Write Access read/write
LV Status available
# open 0
LV Size 7.90 GiB
Current LE 2023
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0

--- Logical volume ---
LV Name /dev/dataarray/home
VG Name dataarray
LV UUID x93XWk-7Qut-hvZ2-pqhB-Qq7s-YeeC-4ABdWX
LV Write Access read/write
LV Status available
# open 0
LV Size 3.00 GiB
Current LE 768
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1

If you want you can format the array and the LV, but this step can be performed during the installation to do, here are the commands just for completeness:

[root@archiso ~]# mkfs.ext2 /dev/md1
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
24096 inodes, 96256 blocks
4812 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
12 block groups
8192 blocks per group, 8192 fragments per group
2008 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

[root@archiso ~]# mkswap /dev/md2
mkswap: /dev/md2: warning: don't erase bootbits sectors
on whole disk. Use -f to force.
Setting up swapspace version 1, size = 996012 KiB
no label, UUID=1bc5ded3-8ef6-4e3a-9d1b-6f6b146d05fc

[root@archiso ~]# mkfs.ext4 /dev/dataarray/home
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
196608 inodes, 786432 blocks
39321 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=805306368
24 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912

Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 23 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@archiso ~]# mkfs.ext4 /dev/dataarray/root
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
518144 inodes, 2071552 blocks
103577 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2122317824
64 block groups
32768 blocks per group, 32768 fragments per group
8096 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

Now you can proceed with the installation of the distribution. Type

[root@archiso ~]# /arch/setup

We proceed following the steps provided by the installer: Select the installation source (in this case the CD), we set the time zone (optional), and goes to the preparation of discs (Prepare hard drive).
The preparation of the hdd must be done manually (option 3 Manually configure block devices and filesystems and mountpoints), select the option to take the device directly from /dev and then set the filesystem (fs), a potential label and mount point :

  • /dev/md1 select ext2 as fs and /boot as mount point
  • /dev/md2 as swap partition
  • /dev/mapper/dataarray-home select ext4 as fs and /home as mount point
  • /dev/mapper/dataarray-root select ext4 as fs and / as mount point

After the disk management selecting the software to install (dase + base-devel) and verify that at least for the raid and lvm packages are marked for installation (lvm e mdad).

Before performing the steps to configure the system open another shell by pressing ALT+F2 to re-create the configuration file of the raid (/etc/mdadm.conf)

[root@archiso ~]# mv /mnt/etc/mdadm.conf /mnt/etc/mdadm.conf.bk
[root@archiso ~]# mdadm --examine --scan >> /mnt/etc/mdadm.conf

press ALT + F1 to return to the installation and configure the system:
on file /etc/mkinitcpio.conf add mdadm and lvm2 to line HOOKS before the filesystem

HOOKS="base udev autodetect pata scsi sata mdadm lvm2 filesystems"

on file /etc/rc.conf set

USELVM="yes"

and finally in /etc/fstab check mount points.
These are fundamental changes, then clearly you can run all those who serve you as the demons start-up network settings, root password etc.

Now through the installer you can install the bootloader and the installation is completed.
That remains is to reboot, and enjoy the system in RAID and LVM.

To verify that everything went well so it is possible, after the first reboot and waiting that the arrays are synchronized, remove a disc and make sure your system is always able to start and be used. Done this you can proceed with the detachment of the other disk, but after placing the first and rebuilt the array.
To add a disk or partition to an array just run

[root@araida ~]# mdadm --add /dev/md3 /dev/sda3

this place into md3 array the partition sda3.

If grub is installed only on a disc, it can run on other typing

[root@araida ~]# grub
grub> device (hd0) /dev/sdb
grub> root (hd0,0)
grub> setup (hd0)
grub> quit

AND THAT’S ALL FOLKS

Resume work!!!

posted by onTuesday February 15th, 2011

Hello folks

As you can see the site and the community have remained almost stationary or for quite some time, as usual causes are more varied, either lack of time and so little desire etc…

Right now we have some taken over the site, we have updated wordpress to the latest version, re-wrote the theme and installed qtranslate to try to give more visibility to the site, also in the international arena.

As for the theme it has been written almost entirely by exploiting the features of CSS3 and using very few images (one or two): the logo has changed now we have the penguin in Ghostbusters style (which to be honest we like more than the previous). The layout is similar to the previous: two sides menu plus one just below the head.

The translation of the site (in English) has just begun: for now have been translated only the homepage and the section collaborate, but in the coming days you will find other pages translated and hopefully a few articles. If anyone has the time / desire to help in the translation are welcome.

It’s time to get back to work and try to bring back the old Spaghettilinux levels.

People let’s give prestige to our community.

Recover deleted files

posted by onFriday December 10th, 2010

Può capitare di cancellare (per errore?) dei file importanti e questo è causa (se non si ha un backup) di molta frustrazione e talvolta (i file importanti? del capo) può portare ad essere vittime del bunga bunga (lascio a voi scoprire di cosa si tratti); per fortuna esistono vari tools atti al recupero di questi dati tra cui testdisk e autopsy.

Quando ci si trova a dover recuperare un file cancellato è molto importante non usare più il supporto fino a quando non si sia recuperato il file, questo perchè il file è recuperabile fintantochè non si è andati a “riscrivere” (mica voi di vostra spontanea volontà ma lo fa il sistema) sulla stessa porzione di disco (o parte di essa): quando un file viene “deletato” (in maniera normale) viene semplicemente eliminato il puntatore alla sua locazione nella tabella delle partizioni.

Detto ciò come procedere?

La prima cosa da fare è una copia del supporto (disco, penna usb ecc) interessato per non corre il rischio di continuare a fare danni (come se quelli già fatti non siano abbastanza), per creare la copia del disco è necessario avere a disposizione un altro supporto con spazio libero almeno quanto la capienza del disco che vogliamo recuperare e un po’ di tempo, tempo che varia a seconda delle dimensioni del supporto.

Precedentemente alla copia del disco si crei un file (di testo) che poi verrà cancellato, questo sarà il file che sarà poi ripristinato:

$touch /media/PENNEGAUSB4/prova_recupero_file && echo -e "\n Ecco il file che verrà cancellato per tentarne il ripristino. \n Speriamo bene :D" >> /media/PENNEGAUSB4/prova_recupero_file

$cat /media/PENNEGAUSB4/prova_recupero_file 

 Ecco il file che verrà cancellato per tentarne il ripristino.
 Speriamo bene 

$ls  /media/PENNEGAUSB4/
altro_file  prova_recupero_file

$rm /media/PENNEGAUSB4/prova_recupero_file
$ls /media/PENNEGAUSB4/
altro_file

Il file non viene più listato quindi non c’è (o meglio non è visibile).
La copia del disco può essere eseguita (dopo aver smontato il disco per garantire che nulla venga scritto nel frattempo) tramite il comando dd, un esempio, supponendo che il disco da copiare sia /dev/sdb, è:

$dd if=/dev/sdb conv=sync,noerror of=disk_img.dd
8060928+0 record dentro
8060928+0 record fuori
4127195136 byte (4,1 GB) copiati, 271,547 s, 15,2 MB/s

Ora che l’immagine è pronta bisogna verificare che questa ed il disco siano la stessa cosa, per farlo basta usare l’eseguibile md5sum e verificare che gli hash coincidano

$md5sum /dev/sdb
4c810b0dc6847520cfe1a092a1eaad7f  /dev/sdb

$ md5sum disk_img.dd
4c810b0dc6847520cfe1a092a1eaad7f  disk_img.dd

Adesso occorre prelevare qualche informazione sul disco che servirà per far funzionare correttamente testdisk

$fdisk -lu disk_img.dd 

Disco disk_img.dd: 4127 MB, 4127195136 byte
197 testine, 15 settori/tracce, 2727 cilindri, totale 8060928 settori
Unità = settori di 1 * 512 = 512 byte
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Identificativo disco: 0x000437c9

 Dispositivo Boot      Start         End      Blocks   Id  System
disk_img.dd1            2048     8060927     4029440    b  W95 FAT32

Le stesse informazioni vengo fornite se fdisk viene eseguito sul device (se gli hash coincidono è come se si lavorasse sullo stesso device)

$ fdisk -lu /dev/sdb

Disco /dev/sdb: 4127 MB, 4127195136 byte
197 testine, 15 settori/tracce, 2727 cilindri, totale 8060928 settori
Unità = settori di 1 * 512 = 512 byte
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Identificativo disco: 0x000437c9

Dispositivo Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048     8060927     4029440    b  W95 FAT32

Quello che interessa è la geometria del disco (CHS dall’inglese cilindri testine e settori) che poi si deve settare su testdisk.
Non rimane che lanciare il programma di recupero:

$testdisk disk_img.dd

Si aprirà la schermata di testdisk con evidenziato il disco che gli è stato passato come parametro. Non si faccia caso all’avviso che appare in rosso (le operazioni che seguono possono essere eseguite anche da un normale utente, diverso se si lavorasse sul device) .
Per iniziare la procedura di recupero spostarsi con i tasti freccia su “PROCEED” e dare invio.

Selezionare ora il tipo di partizione (solitamente INTEL o NONE se si vuole controllare una partizione) e premere invio.
Successivamente si setta la geometria del disco selezionando la voce “Geometry” ed impostando i parametri che si sono appresi con il comando fdisk, per poi selezionare la voce “Analyse” per far partire la procedura di recupero.

Nella schermata successiva selezionare la voce “Quick Search”, successivamente all’individuazione della partizione (o delle partizioni) basta premere “P” per accedere ai file del disco. I file cancellati verranno mostrati in rosso.

Selezionare il file che si vuole ripristinare, premere “C” per salvarlo, selezionare il path dove scrivere il file e premere “Y” per copiarlo localmente. Ripetere queste operazione per tutti i file che si voglio recuperare.

Terminato di “ripristinare” tutti i file premere “Q” per uscire, poi invio ed ancora selezionare “QUIT” fino a quando non si chiude il programma.
Adesso è possibile godersi il file recuperato, andare a letto tranquilli ma soprattutto senza aver subito il bunga bunga da parte del capo…

$ls
disk_img.dd  guidarecuperodati  img_usbpenn.dd  prova_recupero_file  vol2-C...OrphanFiles.20060720.PDF  wifi

$ cat prova_recupero_file 

 Ecco il file che verrà cancellato per tentarne il ripristino.
 Speriamo bene 

Root password lost?

posted by onTuesday May 25th, 2010

Are you trying to access your system as root but can not remember your password? You are about to fall into despair because you believe you have to do the impossible to recover your administrator password? Thinking about having to reinstall the system?ALT Stop here and continue reading.

The following procedure resets the password to the root user, the only thing to do is add/modify some parameters to pass to the system at boot time and then assumes you have access to the machine.

At the grub prompt, select the row to edit and press “e” as “EDIT”, to the next screen will display the lines that run when you select the “TITLE” selected in the previous step. Now using the arrow keys to select the line that contains something like:

kernel /vmlinuz26 root=/dev/sda3 ro radeon.modeset=0 vga=791

press again “e”to edit and change the line

kernel /vmlinuz26 root=/dev/sda3 rw  init=/bin/bash radeon.modeset=0 vga=791

Complete the changes just press enter and then “b” to start the system., at the end of the start-up we will be in a root schell, at this point just type the command passwd for change the password.

MOCA 2008

posted by onThursday June 12th, 2008

Dal 21 al 24 Agosto 2008, presso il Parco “ex Caserma Di Cocco” (Pescara), si svolgerà la seconda edizione del Metro Olografix CAmp, un hacker camp in stile nord-europeo, ad accesso libero e gratuito.

L’evento si è già svolto 4 anni fa, ed ha riscosso una significativa participazione di pubblico, non solo in termini numerici ma anche in termini di coinvolgimento. Le parole chiave sono Pescara, Tenda e PC.

Dalle premesse anche l’evento di quest’anno sembra avere le stesse caratteristiche. Per chiunque volesse partecipare attivamente, gli organizzatori accettano suggerimenti sui talk, aree tematiche o altre iniziative entro e non oltre il 1 di agosto.

E’ sufficiente scrivergli, e tutti i riferimenti sono reperibili sul loro sito: http://camp.olografix.org