While exploring other options for easily converting Jupyter notebooks along with pandas DataFrame tables to markdown, I came across nbdev which is used to create delightful python projects using Jupyter Notebooks.

nbdev is a system for exploratory programming. It is a library created by the fast.ai team that allows you to develop a Python library in Jupyter Notebooks, putting all your code, tests and documentation in one place. nbdev

I’m not sure if I will be publishing Python packages quite yet but it could be a handy way of organising code instead of copying and pasting functions etc from one notebook into another. Maybe I can also use it for converting the notebooks locally and incorporating into a Hugo site such as this.

nbdev

• nbdev docs
• A Tutorial for a step by step guide
• a Minimal Example

I followed the tutorial which involved the following steps: (It looks like there might have been a few small layout changes to GitHub since the tutorial was written but it is easy to navigate around).

1. Installing nbdev using either pip or conda.
2. Create a repo using the nbdev template.

• The name of your project will become the name of the Python package generated by nbdev and therefore it is recommended to pick a short, all-lowercase name with no dashes between words (underscores are allowed)

• The nbdev system uses jekyll for documentation which is supported by GitHub pages. Therefore the site can be hosted for free on Github pages without any additional setup although any other jekyll hosting can also be used.

  • Click Settings in the repo and go to the GitHub Pages setting which has its own tab now.
  • Set the Source for the GitHub Pages site to be built from the docs folder in the master branch.
  • Copy the URL. (The field will initially be on a blue background until the site is published, then it will have a green background).
  • Go back to the main repo page and click the settings icon on the right beside the About and paste the URL into Website field. The URL is likely to be there already but grayed out. You can also add a Description and some Topics.

Note: Almost immediately after creating a repo using the nbdev template, and before getting a chance to go to next step and edit the settings.ini file I get notified that all jobs have failed. CI workflow run. I have tried editing this file on GitHub before cloning to local repo and I still get this message immediately. However it does resolve as I progress through the next steps.

3. Open terminal and clone the repo created above.

4. Edit the settings.ini file.

The tutorial outlines the settings that need to be given values including the lib_name, user, description, keywords, author, author_email, copyright and branch. These are all initially commented out using as # in the settings.ini file. I set the lib_name as the name of the repo. (There is a repo_name and company_name to be completed for Enterprise Git). The user is your GitHub user name and branch is master. I think it could also be main. The copyright value must also be supplied which is your name or company name as well as an author_email. Without these values you will get errors.

There are some other optional parameters in the settings.ini file which I left be.

5. Install git hooks to avoid and handle conflicts

Jupyter notebooks may not always work seamlessly with git. I haven’t encountered this yet. Run nbdev_install_git_hooks in the terminal from the project folder. This sets up git hooks which will remove metadata from the notebooks when you commit and reducing the chances of conflicts.

6. Edit 00_core.ipynb

The next step is to launch jupyter notebook and select this file. Add a title and summary of your module. The tutorial gives an example and explains what is required here.

• If you include a comment #export in a python code cell this means the code in that cell will be included in the module and documentation.

• Tests can be included (using assert).

• Markdown headings will be used to create a table of contents in the documentation.

7. Build the library

Run nbdev_build_lib from the terminal (from anywhere within the project folder). This should create the python module.

Converted 00_core.ipynb. Converted index.ipynb.

8. Edit index.ipynb

This step creates the documentation home page and README.md file from the index.ipynb file.

#hide
from your_lib.core import *

Edit the first line of index.ipynb to use the name you used in the settings.ini file instead of your_lib.

#hide
from test2.core import *

Add a project name and description in the markdown cell and add some instructions on how to use the module, including some code examples.

9. Build the docs.

Run nbdev_build_docs from the terminal (within the project folder). This exports HTML versions of the notebooks to the docs directory. It creates hyperlinks for any words in backticks (that exist in your module).This will also create a menu for all the notebooks you created and a table of contents for each.

• Add in-notebook export cell There is also an option to export all the modules directly from a notebook instead of having to go to the terminal to do it. Add this line to any cell and run it to export your modules. It is recommended to be placed in the last cell of the notebook.

from nbdev.export import notebook2script
notebook2script()

• Run tests in parallel.

You can optionally run tests before pushing to GitHub by running nbdev_test_nbs in the terminal.

10. Commit to GitHub.

using git add, git commit and git push commands. If everything is in order then after a few seconds or more you should be able to see the site published on Github pages at the website link under About.

References

• nbdev
• nbdev tutorial