Jupyter Notebook. portal.biohpc.swmed.edu

Transcription

1 Jupyter Notebook [web] [ ] portal.biohpc.swmed.edu

2 What is Jupyter Notebook? Jupyter Notebook is an interactive web application that allows users to: Interactively develop code Create publication quality figures Produce documented workflows and pipelines Use cases at UT Southwestern: Teaching courses Machine learning Statistical modeling 2

3 Example Bioinformatics Workflow 3

4 Jupyter Notebook Central Concepts Code Environment Kernels Markdown Magics 4

5 So how does it work? Jupyter Server on a Nucleus node Notebook File HTTP & Websockets Web Browser 5 work.html

6 Default Jupyter Environment only has base Python 3.6 environment kernel? Default Environment 6

7 Conda Requires a Conda environment Install R, Python, or Julia kernel packages Load into Jupyter Notebook 7

8 Adding Different Python Environments into Jupyter Notebook Installing Python 2.7 kernel into Jupyter Notebook $ module load python/2.7.x-anaconda $ ipython kernel install --user Installed kernelspec python2 in /home2/s178337/.local/share/jupyter/kernels/python2 Installing a custom conda environment into Jupyter Notebook, e.g. Python 3.6 with DeepTools $ module load python/ anaconda $ conda env list # conda environments: Python env requires the ipykernel package to be installed! R_env py3.6-deeptools root /home2/s178337/.conda/envs/r_env * /home2/s178337/.conda/envs/py3.6-deeptools /cm/shared/apps/python/ anaconda $ ipython kernel install --user --name py3.6-deeptools --display-name "Python 3 DeepTools Installed kernelspec py3.6-deeptools in /home2/s178337/.local/share/jupyter/kernels/py3.6-deeptools 8

9 Installing an irkernel into Jupyter Notebook $ conda env list # conda environments: R_env py3.6-deeptools root Your R env requires the irkernel package to be installed! /home2/s178337/.conda/envs/r_env * /home2/s178337/.conda/envs/py3.6-deeptools /cm/shared/apps/python/ anaconda $ source activate R_env (R_env) $ R R version ( ) -- "Single Candle" Copyright (C) 2017 The R Foundation for Statistical Computing Platform: x86_64-pc-linux-gnu (64-bit) > Irkernel::installspec() > [InstallKernelSpec] Installed kernelspec ir in /home2/s178337/.local/share/jupyter/kernels/ir > q() 9

10 How do I remove a kernel? User kernels are stored inside ~/.local/share/jupyter/kernels $ ls ~/.local/share/jupyter/kernels/ ir py3.6-deeptools python2 python3 $ rm -rf.local/share/jupyter/kernels/py3.6-deeptools/ Your conda environment is still installed. Don t worry! 10

11 Markdown Syntax What is Markdown? A lightweight markup language with plain text formatting syntax that is becoming a new standard for scientists and academics for workflows, teaching, and code documentation. What does that mean for writers? Markdown allows the writer to quickly create rich text that can be converted to HTML with a simple plain-text editor. Markdown is natively supported within Jupyter notebooks. 11

12 Markdown Syntax Rendered Markdown # An exhibit of Markdown This note demonstrates some of what [Markdown]( projects/markdown/) is capable of doing. *Note: Feel free to play with this page. Unlike regular notes, this doesn't automatically save itself.* ## Basic formatting Paragraphs can be written like so. A paragraph is the basic block of Markdown. A paragraph is what text will turn into when there is no reason it should become anything else. Paragraphs must be separated by a blank line. Basic formatting of *italics* and **bold** is supported. This *can be **nested** like* so. 12

13 Markdown Syntax -- Resources How can I learn Markdown syntax? Recommended Cheat Sheet: Learn it interactively! Need a helping hand? Try MarkdownPad! 13

14 Markdown Syntax Jupyter Notebook Select Markdown in the drop-down menu Then hit Shift + Enter or click the Play button to render the cell. 14

15 Markdown Syntax Jupyter Notebook The Markdown code is now rendered. 15

16 Markdown Syntax and HTML HTML can also be rendered in Markdown cells. 16

17 17

18 LaTeX Support LaTeX math is also supported in Markdown cells and rendered via MathJax. \begin{eqnarray*} e^x &\approx& 1+x+x^2/2! + \\ && {}+x^3/3! + x^4/4! + \\ && + x^5/5! \end{eqnarray*} One can also produce a full LaTeX document from a Notebook -- WIP. 18

19 ipython Going beyond the standard Python The four most helpful commands 19

20 ipython Going beyond the standard kernel Magic commands ipython includes OS-like shell commands call magics. Line magics %command Cell magics %%command Rest of the line after magic call is received as the argument. Arguments are not only the rest of the line, but the rest of the cell. Type %lsmagic for a list of available magic commands. 20

21 ipython Examples Demo ipython_examples.ipnb? operator Line magics Cell magics OS shell commands 21

22 Exporting a Jupyter Notebook $ ipython nbconvert --to FORMAT notebook.ipynb 22

23 Exporting a Jupyter Notebook Example: Exporting a Notebook to HTML $ ipython nbconvert --to html notebook.ipynb Example: Exporting a Notebook to PDF $ ipython nbconvert -to pdf notebook.ipynb Example: Exporting a Notebook to a Python script $ ipython nbconvert -to script notebook.ipynb 23

24 Coming Soon! Run GPU-accelerated code directly within a Jupyter Notebook! Example: Tensorflow-GPU workflows 24