How to use `stackyter` to run JupyterLab notebooks¶

This tutorial presents step-by-step instructions to execute Jupyter Notebooks configured to use the LSST software framework on hosts in CC-IN2P3 Login Farm. It is an alternative, older method to the one presented in How to execute LSST-enabled JupyterLab notebooks, which we recommend. However, both methods work.

A Jupyter Notebook is a web application that allows you to create documents that contain code (in particular Python code which uses the LSST software framework), narrative text, visualizations, etc. It is also a convenient way to explore data and document your work. You can find more information in the Jupyter project web site.

By executing Jupyter Notebooks at CC-IN2P3 you can explore the datasets interesting for LSST using the LSST software framework, without you needing to install on your personnal computer neither the LSST software nor the datasets.

Before being able to open a notebook, you need to:

Install Python on your personal computer
Install stackyter on your personal computer
Configure stackyter
Configure your SSH client

This installation is a one-time process.

Step 1: install Python on your personal computer¶

You need a working installation of Python on your personnal computer. We recommend the Anaconda distribution of Python 3.

Step 2: install stackyter on your personal computer¶

stackyter is a convenience tool to make easier executing Jupyter notebooks at a remote host, for instance at hosts in CC-IN2P3’s Login Farm. To install it on your personnal computer do:

pip install --upgrade stackyter

Further details and alternative installation options can be found in stackyter‘s documentation. Once installed we recommend you take a few minutes to configure it for your own needs as presented in the next step.

Step 3: configure stackyter¶

On your personal computer create a configuration file for instructing stackyter how to operate. By default, the location of stackyter‘s configuration file is $HOME/.stackyter-config.yaml. An example of such a file is shown below:

$ cat $HOME/.stackyter-config.yaml
{
   # Use the configuration named 'ccin2p3' by default. Useful if you have several
   # configurations for various remote servers in the same configuration file.
   'default_config': 'ccin2p3',

   # Configuration for using the LSST stack at CC-IN2P3
   'ccin2p3': {
        # Host where runs the Jupyter server. You can use a fully qualified host name
        # (e.g. cca.in2p3.fr) or a 'Host' entry of your SSH configuration file
        # (typically located at $HOME/.ssh/config)
        'host': 'ccjupyter',

        # Flavor of Jupyter to use. Valid values: 'lab' or 'notebook'
        'jupyter': 'lab',

        # Absolute path of the directory at the remote host (i.e. at CC-IN2P3) where you
        # want Jupyter to save your notebooks.
        # NOTE: replace 'youruser' by your login name
        'workdir': '/pbs/throng/lsst/users/youruser/notebooks',

        # Execute this list of commands before launching Jupyter. Use this to setup
        # the specific version of the LSST stack you want to use and to setup the
        # EUPS products you need in your work session.
        'runbefore': [
            "source /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/w_2022_12/loadLSST-ext.bash",
            "setup lsst_distrib"
        ],
   }
}

Important

You need to adapt this sample configuration for your use case. Specifically, in the workdir entry you need to replace youruser by your login name at CC-IN2P3 or to change the path of the directory where you want Jupyter to store your notebooks.

In addition, in the runbefore entry, you may want to replace w_2022_12 by another release of the LSST science pipelines. To know what releases are available, log in to the Login Farm and do:

ls -l /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib

There are other configuration options may find useful for your particular use case. You can find them all in stackyter official documentation. There are also some examples of configuration files.

Step 4: configure your SSH client¶

To configure and launch Jupyter at the remote host, stackyter uses the SSH client present in your personnal computer. In particular, in the configuration file above we used a host named ccjupyter which needs to be present in the SSH client configuration file. Add the entry below to your SSH configuration file (typically $HOME/.ssh/config):

Host ccjupyter
   Hostname cca.in2p3.fr
   User     youruser

If you have configured your computer to connect to CC-IN2P3 Login Farm without typing your password at every connection (see How to customize your SSH client), you may want instead to add the entry below to your SSH configuration file:

Host ccjupyter
   Hostname                  cca.in2p3.fr
   User                      youruser
   GSSAPIClientIdentity      youruser@CC.IN2P3.FR
   GSSAPIAuthentication      yes
   GSSAPIDelegateCredentials yes
   GSSAPITrustDns            yes

Important

Replace youruser above by your login name at CC-IN2P3.

Please note that you can add other options to the ccjupyter host entry of the SSH configuration file as you need. Those options will automatically be used by stackyter when establishing the connection and launching Jupyter.

Launch Jupyter¶

Now you are ready to launch Jupyter by executing stackyter on your personnal computer:

stackyter.py

Upon successfull establishment of a SSH connexion to the remote host and launch of Jupyter, stackyter displays serveral messages. You will see appear one highlighted message similar to the one below:

...
Copy/paste this URL into your browser to run the notebook locally
   http://localhost:20001/?token=7722079686b74117ef2f6cf7b425f91c8cf6cad34086f409

Point your web browser to the URL shown in the message. Please note that this URL is unique and changes every time you launch Jupyter.

You can now interact with your remote notebook. From within your notebook you can access the datasets of interest for LSST (see Datasets: /sps/lsst/datasets).

Once you are done, type Control-C to stop the remote Jupyter server and automatically close the underlying SSH connection to the remote server that stackyter established.

Troubleshooting¶

stackyter needs to be regularly adapted to be compatible with the evolving Jupyter command it relies on. If you have trouble making it work for you, please ensure that you have the most up-to-date version of it installed on your personnal computer. The source of truth is stackyter‘s source code repository

How to use stackyter to run JupyterLab notebooks¶