In this post, I am sharing how I set up my Arch Linux system for machine learning and development. If I boot up a fresh arch install I do these steps and set up the machine for my work and productivity. There are very few guides to set up a development and machine learning arch Linux based environment and I hope this post will guide you to set up your Arch Linux working environment. Softwares and extensions which I installed in this post can be tweaked based on your needs and preferences. All the below steps works well for me. If you are starting your Arch journey then you can start with Manjaro which is an arch based distro which I am personally using as my daily device.
1
sudo pacman-mirrors --fasttrack
1
sudo pacman -Syyu
yay(yet another yogurt) is an AUR helper that lets to install a new package from AUR, update and remove installed AUR packages.
1
2
3
4
5
6
pacman -S --needed base-devel
git clone <https://aur.archlinux.org/yay.git>
cd yay
makepkg -si
cd ..
rm -rf yay
If you are in power restricted devices like laptops, Linux might not give the battery backup that you expected. Most CPUs are not power-optimized for Linux, installing this package will automatically adjust CPU speed & power based on usage. It allows us to improve battery life without making any compromises. We do not have to configure anything just install it and see the difference. It’s a must install tool on my laptop.
1
2
3
sudo pacman -S auto-cpufreq
systemctl enable auto-cpufreq
systemctl start auto-cpufreq
Now install all your productivity applications, favourite applications, IDE’s, code editors, Spotify etc. With the power of AUR, all these things are in your reach, no need to hop around just search AUR and find your application.
In fresh installation, I install these applications:
1
yay -S chrome brave-bin slack-desktop discord spotify kdeconnect vim
I install miniconda from its website instead of AUR as it already set up all things automatically also miniconda package I once installed from AUR was conflicting with python installed.
1
2
3
4
cd Downloads
wget <https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh>
sudo chmod +x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh
now proceed to the installation process and now you have conda setup just restart the terminal and check conda version if its working or not. By default installing this way will add the path to the .bashrc file.
1
conda --version
Installing vscode
1
sudo pacman -S vscode
By default I could not access vscode marketplace for extensions it always says we cannot connect to extension marketplace. I installed code-marketplace package from AUR and it fixed the issue.
1
yay -S code-marketplace
Next, I install language extensions for C, C++, Python and Rust other language extensions I add based on the use case.
Next, I install my favourite vscode extensions for productivity, these extensions I add anytime when setting up freshly installed vscode.
With the installation of miniconda, when we open a new instance of the terminal base environment is activated by default which I do not like. I want to activate conda environment when I needed them. We can disable this default behaviour with the following command.
1
disable default activation of base environment: conda run conda config --set auto_activate_base false
1
2
sudo pacman -S nvidia nvidia-utils
If you are using manjaro with proprietory drivers then no need to do the above step as Nvidia drivers are already installed.
1
2
nvidia-smi
We can install packages using pip or conda based on preference. PyTorch has very good installation instruction and is easy to setup just use latest installation instruction from its website for conda and pip. For TensorFlow we can use conda which will install all dependencies along with cuda and cudnn but we might not get latest version of library. I prefer installing TensorFlow using pip because conda TensorFlow packare is outdated and I was not able to create tensors in GPU while creating tensors the process gets stuck without any errors. Installing TensorFlow via pip require some additional steps.
1
conda create --name envname
1
conda activate envname
1
conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch
for latest instructions visit pytorch official site.
Using conda
1
conda install tensorflow-gpu
Using pip
With this method we have to also install cuda-toolkit and cudnn these packages can be found in pacman.
1
sudo pacman -S cuda-toolkit cudnn
After installing restart the system and then verify cuda install
1
nvcc -V
Now activate virtual conda environment and install Tensorflow using pip
1
2
conda install pip
pip install tensorflow-gpu
We can verify detection of cuda by TensorFlow and PyTorch using a small python code snippet and check its output
PyTorch
1
2
import torch
print(torch.cuda.is_available()) # should be True
TensorFlow
1
2
import tensorflow as tf
print(tf.config.list_physical_devices("GPU")) # should be list of GPUS
Docker can be found in community repo of archlinux and can be installed using pacman.
1
sudo pacman -S docker
Now we have to enable and start docker service
1
2
sudo systemctl start docker.service
sudo systemctl enable docker.service
Currently we need to run docker commands with sudo and root password to solve this we need to add current user account to docker group
1
sudo usermod -aG docker $USER
Install Docker compose
1
sudo pacman -S docker-compose
After this command reboot the machine. After rebooting check if docker runs succesfully by running docker hello world.
1
docker run hello-world
I have my website build using jekyll, so I need jekyll to test changes to website in local before deployment.
1
2
sudo pacman -S ruby
1
2
gem install jekyll
1
2
3
export GEM_HOME="/home/username/.local/share/gem/ruby/3.0.0/gems/"
export PATH="/home/username/.local/share/gem/ruby/3.0.0/bin/:$PATH"
Rust is installed and managed by the rustup tool it makes the process easier to install and manage rust versions.
1
curl --proto '=https' --tlsv1.2 <https://sh.rustup.rs> -sSf | sh
Github has now removed support for password-based authentication for git operations, so accessing git through the command line will not be possible after August 2021. Github will no longer accept account passwords when authenticating Git operations on GitHub.com. We have to use a token-based authentication or access via ssh. I use ssh based authentication for git operations in Github.com.
1
2
- ssh-keygen -t ed25519 -C "your email here"
upload public key to github > settings > ssh and gpg keys. Public key will have pub extension just copy the contents of that file and paste in the textbox provided.
adding ssh key to ssh-agent
1
2
ssh-add .ssh/generated_keyname
1
2
ssh -T [email protected]
it will print out your username in console.
If you are using arch desktop in dual boot settings with windows then time can change if you hop between different boots. The below command will enable RTC convention instead of UTC for time and date in linux which windows uses by default.
1
timedatectl set-local-rtc 1 --adjust-system-clock
If you have system with more than one hard drives, like in my case I have two hard drives a M.2 SSD in which linux is installed and a mechanical hard drive for backup and media contents. By default linux will not mount hard drive or partitions automatically other than swap, root and home, so some applications that rely on these drives for saving and loading data will fail on startup as they will not able to access that location. I usually auto mount my backup and media drives to load my music automatically configure backup at startup.
1
2
sudo mkdir /media
1
2
3
sudo mkdir /media/multimedia
sudo mkdir /media/backup
I want to mount my backup drive in /media/backup and multimedia drive in /media/multimedia
1
2
sudo fdisk -l
1
2
sudo blkid
1
2
sudo nano /etc/fstab
The entry we have to add in the file should be of form
UUID="PASTE HERE"<tab>/media/folder_name<tab>filesystem<tab>defaults<tab>0<tab>0
here is an example
1
UUID=01D4653A0E0898E0 /media/multimedia ext4 defaults 0 0
For more info about editing fstab file and details of different parameters visit this link
If you are mounting windows ntfs drive, it might not give write permissions to that drive, to solve this issue we have to follow some additional steps.
Disable fast startup in windows
The option can be found inside Control Panel > Power Options > Choose what the power button do > Turn off fast startup
Install ntfs-3g
1
2
sudo pacman -S ntfs-3g
1
2
sudo blkid
this command lists all drive partitions in the device, from this select the ntfs drive to enable write access.
1
2
sudo ntfsfix /dev/sda3
replace /dev/sda3 with your partition.
We can also automount windows ntfs partition with the help og ntfs-3g that will also enable read and write in that partition. The syntax for this is same as for mounting ext4 filesystem, just replace paritition filesystem with ntfs-3g in the above syntax, here is an example
1
UUID=0CC84430C84419FA /media/data ntfs-3g rw 0 0
MS fonts like ARIAL, Times of Roman etc. can be easily installed using a single command.
1
yay -S ttf-ms-fonts
By default linux will not display emoji instead it will display a black square. To enable emoji support follow steps below.
1
2
yay -S noto-fonts-emoji
1
2
sudo nano /etc/fonts/conf.d/emoji.conf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
<alias>
<family>sans-serif</family>
<prefer>
<family>Your favorite sans-serif font name</family>
<family>Noto Color Emoji</family>
<family>Noto Emoji</family>
</prefer>
</alias>
<alias>
<family>serif</family>
<prefer>
<family>Your favorite serif font name</family>
<family>Noto Color Emoji</family>
<family>Noto Emoji</family>
</prefer>
</alias>
<alias>
<family>monospace</family>
<prefer>
<family>Your favorite monospace font name</family>
<family>Noto Color Emoji</family>
<family>Noto Emoji</family>
</prefer>
</alias>
</fontconfig>
If you want to use emoji then installing a emoji picker will help.
1
yay -S ibus-uniemoji
ibus emoji and set your favourite key combination for opening emoji picker. I am using Alt + . as shortcut.