1.Setting up File Server
In distributed systems a files server is one (or many) hosts attached to a network and providing a file storage service that the workstations or remote clients can use. The interaction between the client and server in this case is organized by the file server protocol. Multiple protocols exist in the domain of file services and can be categorized into groups:
- LAN side vs. Internet side protocols
- LAN side protocols usually offer tighter bindings between the server side storage and client side. The client in this case has the remote storage explicitly visible as a local file system ( we call it mapping or mounting the remote file system). The protocols here: NFS and CIFS/SMB. In addition to FS-level transparency the LAN side protocols usually offer a service discovery helping the clients to discover the file services in the scope of LAN.
- Internet side protocols are mostly reduced to request/response protocol design (FTP,SFTP,HTTP) and therefore are focusing on download,upload,delete etc. primitive actions (as opposed to NFS where we can edit the file directly - in FTP we have to download the file first).
- A special subset here are protocols like DropBox and OwnCloud (and SSHFS) which are in fact Internet side protocols but do offer remote directory mapping (to local file systems). However when editing the file on the mapped directory - the changes are stored locally first and then synced to the remote storage by the corresponding client software. Here the software on client side is determined to watch for the changes on both remote and local files and sync them correspondingly).
- Standalone vs. Network File System vs. Distributed File System
- How the actual data is stored:
- on single server standalone
- on multiple servers hierarchically
- on multiple servers with block-level redundancy
- How the actual data is stored:
In case of NFS and CIFS/SMB the redundancy is usually achieved by having different physical servers contributing to different portions of so called common file tree. Just like we have different partitions of hard drive contributing to one root tree:
- /dev/sda1 -> /
- /dev/sda2 -> /var
... we can have different NFS and SMB servers contributing to local root tree:
- smb://10.9.8.7/common/opt -> /opt
- nfs://10.9.8.5/common/home -> /home
- nfs://10.9.8.5/common/share -> /usr/share
Here we see clear resource distribution but we do not see redundancy yet as each subtree is still stored on only one physical server. Redundancy in this case can be achieved by applying a distributed file system (like GlusterFS) to aggregate the storages of multiple servers using file-based replicating image.
Finally the systems like HDFS allows as to have automatic block-level redundancy and distribution:
image
!!! Make a backup!
In this lab there is a higher than normal chance that you will break your machine (errors on boot) so you should make a backup.
Before you continue log into ETAIS
https://minu.etais.ee -> Select your project
-> Resources
-> Virtual Machines
-> Click on your machine name
-> Backups
-> Create
File System Quotas
A disk quota is a limit set by a system administrator that restricts certain aspects of file system usage on modern operating systems. The function of using disk quotas is to allocate fair distribution of storage resources and to protect against accidental filling of the file system.
First we need a disk that we can use without breaking things. When we created a machine in ETAIS
we also specified one small separate 1GB
disk. Now its time to start using it. Do not mix vda
and vdb
in this lab otherwise you might break your whole machine and in worst case scenario you break everything thus must start labs from Week 3
.
- Identify correct path and name of your secondary @@1GB disk
# lsblk
- Now we need to configure (format) it before we can use it.
# fdisk /dev/vdb
- hit
o
to create a new empty DOS partition table - hit
n
to add a new partition - hit
p
to select primary type - hit
1
- Partition number (1-4, default 1): - hit
2048
- First sector (2048-2097151, default 2048) - hit
ENTER
if asked Last sector, +sectors or +size{K,M,G,T,P} (2048-2097151, default 2097151): - hit
w
to write table to disk and exit
# mkfs.ext4 /dev/vdb1
# mkdir /mnt/vdb1
# mount -t ext4 /dev/vdb1 /mnt/vdb1
Our manual is based on public more detailed manual How To Enable User and Group Quotas : https://www.digitalocean.com/community/tutorials/how-to-set-filesystem-quotas-on-debian-9
- Install
quota
package - Initialize the user quotas on newly created file system
- Edit the
/etc/fstab
and add mount point to/dev/vdb1
withusrquota
parameter enabled. /dev/vdb1 /mnt/vdb1 ext4 errors=remount-ro,usrquota,grpquota 0 1
- Edit the
- remount the
/mnt/vdb1
file system withmount -o remount /mnt/vdb1
command - Perform a
quotacheck
witquotacheck -cugm /mnt/vdb1
command - Turn on quotas by running
quotaon /mnt/vdb1
- remount the
- Create new user account called
dataguy
(useadduser
command)
Before we continue with setting the quota for our new user. Let's figure out what is its UID
? A UID (user identifier) is a number assigned by Linux to each user on the system. This number is used to identify the user to the system and to determine which system resources the user can access. UIDs are stored in the /etc/passwd
file. Groups in Linux are defined by GIDs (group IDs). Just like with UIDs, the first 100 GIDs are usually reserved for system use. The GID of 0 corresponds to the root group and the GID of 100 usually represents the users group. GIDs are stored in the /etc/groups
file.
Let's first figure out what is current UID
of a newly created user dataguy
.
Lets have look at current dataguy
UID and GID
# id dataguy
Later in this Lab we are going to setup NFS server and export file system for remote usage. In particular we are going to rely on user dataguy
as NFS expects the UID
will be the same on both NFS server and NFS client hosts. Usually centralized user directory server (DB or LDAP) is in use with NFS to guarantee the uniqueness of user/UID pairs. For our test-setup we just agree we force the dataguy
user to be set with static UID
.
Now we can change dataguy
UID and GID to 2001 (they dont have to match but we like to keep them same)
# usermod -u 2001 dataguy
# groupmod -g 2001 dataguy
Following two command finds all files with old UID
and GID
and changes them to new ones. PS! <...ID> fields needs to be modified with correct parameters.
# find / -user <OLDUID> -exec chown -h <NEWUID> {} \;
# find / -group <OLDGID> -exec chgrp -h <NEWGID> {} \;
i.e. in teacher correct commands looked like this
# find / -user 1005 -exec chown -h 2001 {} \;
# find / -group 1005 -exec chgrp -h 2001 {} \;
Finally lets update dataguy group memberships status.
# usermod -g 2001 dataguy
Output of id dataguy
should like this now.
uid=2001(dataguy) gid=2001(dataguy) groups=2001(dataguy)
After the UID
of a dataguy
user is fixed we may finally proceed to quota settings:
- Set the quotas for
dataguy
with a commandedquota dataguy
- You can exit the editor with
CTRL-K
thenq
followed byy
key combination - Set the
soft quota
of userdataguy
to10MB
(10240kB) - Set the
hard quota
of userdataguy
to20MB
(20480kB)
- You can exit the editor with
Setting the quotas for the user
, root
and mailuser
is not recommended at this moment.
- To test the quotas change user to
dataguy
.- Make a directory at /mnt/vdb1/data
- Move to folder
/mnt/vdb1/data
where quota is active
- Now lets create a big 100MB file ...
- 100MB file can be created by running
$ dd if=/dev/zero of=100MBfile count=1 bs=100M
- How many MB were actually copied? (verify with
ls -lah
the situation)
- 100MB file can be created by running
- The output of
quota -v
command should adequately reflect the situation - Check all user quotas with
# repquota -a
- Delete
100MBfile
sodataguy
quota would not be full.
Installing Samba
Samba is a free software re-implementation of the SMB/CIFS networking protocol, and was originally developed by Andrew Tridgell. Samba provides file and print services for various Microsoft Windows clients and can integrate with a Microsoft Windows Server domain, either as a Domain Controller (DC) or as a domain member. As of version 4, it supports Active Directory and Microsoft Windows NT domains.
Samba runs on most Unix, OpenVMS and Unix-like systems, such as Linux, Solaris, AIX and the BSD variants, including Apple's macOS Server, and macOS client (Mac OS X 10.2 and greater). Samba is standard on nearly all distributions of Linux and is commonly included as a basic system service on other Unix-based operating systems as well. Samba is released under the terms of the GNU General Public License. The name Samba comes from SMB (Server Message Block), the name of the standard protocol used by the Microsoft Windows network file system. Source:Wikipedia.org
- Install samba server and client packages
# apt install samba smbclient
- Main samba configuration file is
/etc/samba/smb.conf
, what contains numerous parameters whose explanation can be read from man pagesman smb.conf
.
# nano /etc/samba/smb.conf
- Most default parameters are very good, but add two new shares by adding a new blocks at the end
[smbshare] comment = Some files from folder /mnt/vdb1/data/rw writable = yes locking = no path = /mnt/vdb1/data/rw guest ok = no browseable = yes [smbreadonly] comment = Some files from folder /mnt/vdb1/data/ro read only = yes locking = no path = /mnt/vdb1/data/ro guest ok = yes browseable = yes
- Now lets create those folders we specified earlier in the configuration.
# mkdir -p /mnt/vdb1/data/rw # mkdir -p /mnt/vdb1/data/ro # chown -R dataguy:users /mnt/vdb1/data/rw # chown -R dataguy:users /mnt/vdb1/data/ro # chmod -R ug+rwx,o+rx-w /mnt/vdb1/data/rw # chmod -R u+rwx,go+rx-w /mnt/vdb1/data/ro
Now lets add user dataguy
to users
group
# usermod -a -G users dataguy
- Samba uses it's own password system so users need to be added by root with a command
# smbpasswd -a <user>
. Note that the users have to exist in/etc/passwd
.
# smbpasswd -a dataguy
- You will be prompted for a password for the user. (It can be different from the one the user has already). For
dataguy
useBiad5up2rzkobslVrdfn
as password. - Restart
smbd
service.
To list existing Samba users:
pdbedit -w -L
- You can verify a samba configuration with a command
testparm
- You can use
smbclient
for testing shares.
$ smbclient -U dataguy //localhost/dataguy
Now lets add Firewall rules to allow other machines to access our shares:
NB! You need to do this only if you did not specify allow-all
as security group during VM creation.
- Use
netstat -utpln
command to check which ports are used bysmbd
andnmbd
service. - Add new security group with name
smb
inETAIS
and add TCP and UDP rules to allow incoming connections to those ports.- Add previously created security group
smb
to yourvirtual machine
- Add previously created security group
Now lets test our samba shares from our personal machine. (You can skip this, if not interested)
- WINDOWS
- Open
File Explorer
or pressWIN+R
and enter\\193.40.154.XX\
(IP of your VM)- You should see two shared folders by default
smbshare
andsmbreadonly
- You should see two shared folders by default
- Accessing
smbreadonly
should work without any prompt of password or user.- Try to create a file in
smbreadonly
(should end in a error Permission Denied)
- Try to create a file in
- Accessing
smbshare
should prompt for a username and password,- In Windows 10 select
More choices
->Use different account
-> username:193.40.154.x\<VM_username>
and Password: <enter correct password> ->OK
- You should see the folder now. Create a testfile there now. (It should work)
- In Windows 10 select
- Go one folder up to
\\193.40.154.x\
level and you should see now new folderuser
(or any other name that your user has) and its content should bereadable
.- The same should work with
\\yourdomain.est\
- The same should work with
- Open
NB! Windows 10 enterprise/education version 1709 and newer disabled guest (anonymous) access to samba shares so you need to enable "insecure" old behavior in Group Policy or registry. Easy way to do it is enter following command in Windows Powershell with Administative rights. Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters -Name AllowInsecureGuestAuth -Value 1
- More info about Windows 10 issue here https://support.microsoft.com/de-ch/help/4046019/guest-access-smb2-disabled-by-default-in-windows-10-server-2016
- In order to see cached network credentials to file shares you can use
net use
command- IN order to delete them you can use command like this
net use \\193.40.154.X\smbshare /delete
- LINUX
- enter
smbclient -U dataguy //193.40.154.X/smbshare
- enter
smbclient -U dataguy //193.40.154.X/user
- More info here
- enter
- MAC OS
Installing NFS
NFS is a network file system protocol allowing a client system access files over a network. The file system is presented to the end-user in a manner similar to a local file system. NFS allows the administrator to share sub-trees of a local file system to the network. The process of sharing is called "exporting".
- NFS Server side
- Update package indexes
- Install packages
nfs-common
andnfs-kernel-server
- Restart service
nfs-kernel-server
- Add
Firewall
rules to enableincoming NEW
traffic toports 111 and 2049
for bothTCP and UDP
- Add new security group with name
nfs
inETAIS
and add TCP and UDP rules to allow incoming connections to those ports. Add previously created security groupnfs
to yourvirtual machine
- Data directories should be already created at
samba manual
/mnt/vdb1/data /mnt/vdb1/data/ro /mnt/vdb1/data/rw
- Assign all previously created directories ownership to user
dataguy
and set group owner tousers
. (we expect you to know how to complete this task without help.) - Edit file
/etc/exports
and add lines
- Assign all previously created directories ownership to user
/mnt/vdb1/data 193.40.154.0/22(rw,sync,fsid=0,crossmnt,subtree_check) /mnt/vdb1/data/ro 193.40.154.0/22(ro,nohide,insecure,sync,subtree_check) /mnt/vdb1/data/rw 193.40.154.0/22(rw,nohide,insecure,sync,subtree_check)
- Repeat this with internal IP (what you see from
ip a
i.e. 192.168.42.XX) - Use command
exportfs -a
to make the newly created shares public
- Repeat this with internal IP (what you see from
- NFS Client side
Client side mounting in our labs can be done in one of 3 ways
- Mounting shares from localhost - good for testing but has no use
- Mounting shares from a friends VM - the way most NFS shares are commonly used, to share data between 2 machines
- Mounting the testshares from student-test.est - this will give you a place to test nfs-client side but not the server side.
Ideally you can find a lab partner to test this with. The procedure for mounting NFS shares looks a little like this.
- We will use package
nfs-common
you installed previously - Use command
rpcinfo -p <vm_name>.est
- Where <vm_name>.est is either your own, or someone else's VM name. (Recommended trying it with other people)
- For testing you can also use student-test.est
You should see:
program vers proto port service 100000 4 tcp 111 portmapper 100000 3 tcp 111 portmapper 100000 2 tcp 111 portmapper 100000 4 udp 111 portmapper 100000 3 udp 111 portmapper 100000 2 udp 111 portmapper 100024 1 udp 33446 status ... many more lines
- Create directories
/media/nfs_ro /media/nfs_rw
- Mount the remote FS using commands:
mount -vvv -t nfs4 <vm_name>:/ro -oro,vers=4,proto=tcp,port=2049,sec=sys /media/nfs_ro/ mount -vvv -t nfs4 <vm_name>:/rw -orw,vers=4,proto=tcp,port=2049,sec=sys /media/nfs_rw/
- Lets test
nfs
client functionality- Change to
dataguy
user - List contents of
ls /media/nfs_ro/
- Try to read some files from
/media/nfs_ro/
- Try to write some files into
/media/nfs_ro/
- Try to write some files into
/media/nfs_rw/
- Change to
---