Python Handbook
Published:
Conda
Install mamba:
conda install mamba -n base -c conda-forge
Remove environment:
conda remove --name r_env --all -y
For MacOS, override your default platform osx-arm64 and replace it with osx-64:
CONDA_SUBDIR=osx-64 mamba create -n renv
mamba activate renv
conda config --env --set subdir osx-64
Tensorflow-gpu in conda (Reference):
export XLA_FLAGS=--xla_gpu_cuda_data_dir=$CONDA_PREFIX
Create a conda environment from an environment.yml file:
conda env create -f environment.yml
Setup of R environment
mamba create -n r_env -y
conda activate r_env
conda config --add channels bioconda
conda config --add channels conda-forge
# conda config --set channel_priority strict
# conda config --set channel_priority false
mamba install r-base=4.1.1 -y
mamba install -c conda-forge -c bioconda r-seurat r-seuratdisk r-signac -y
mamba install -c bioconda bioconductor-genomeinfodbdata -y
mamba install -c bioconda bioconductor-genomeinfodbdata --force-reinstall
Jupyter Notebook
Python kernel
Add a new kernel kernel_name from a conda environment env (Reference):
conda activate env
mamba install ipykernel -y
python -m ipykernel install --name kernel_name
conda deactivate
R kernel
Add a new R kernel kernel_name from a conda environment r_env to another conda environment env:
conda activate r_env
mamba install r-irkernel -y
conda activate env
/raid6/Jinhong/anaconda3/envs/r_env/bin/R -e 'IRkernel::installspec(name="kernel_name", displayname="kernel_name")'
Bash commands
Running scripts
nohup python -u foo.py 0 > foo.out 2>&1 &
nohup Rscript foo.R 0 > foo.out 2>&1 &
Python dependencies
Print the versions of Python packages in the current conda environment:
conda list -f "python|numpy|numba|scipy|pandas|scikit-learn|h5py|anndata|scanpy|statsmodels|matplotlib|joblib|cvxpy|tqdm" | awk '{ print $1" | "$2 }'
R dependencies
Slurm
Check GPU usage on the node n01:
ssh n01 'nvidia-smi -l 1'
Check the status of the jobs:
# all jobs
squeue -u jinhongd
# pending jobs
squeue --start -u jinhongd