What Can `execnb` do?

by Audrey M. Roy Greenfeld | Tue, Dec 24, 2024

This notebook is a [SolveIt](https://solveit.fast.ai/)-style exploration of [https://github.com/AnswerDotAI/execnb/](https://github.com/AnswerDotAI/execnb/). Here I am following the SolveIt process in a Jupyter notebook to learn new things.

Understand the Problem

I've been interested in learning what the open-source execnb package does and how it works. I don't have a particular use case, other than wanting to know more so I can contribute to it or to code that uses it.

Devise a Plan

Carry Out the Plan

Let's see what execnb can do!

from execnb.nbio import *
from execnb.shell import *
from fastcore.all import *
from fasthtml.common import *
from IPython.display import HTML, Markdown
from pathlib import Path

Setup: Let's Grab Some Jupyter Notebooks

We have notebooks in nbs/ in this repo which we can look at.

root = Path('../nbs').parent
nb_dir = root/'nbs'
nb_dir

Yay for fastcore L lists and chainable operations like sorted:

nbs = L(nb_dir.glob('*.ipynb')).sorted()
nbs

Reading a Jupyter Notebook with read_nb

read_nb comes from execnb.nbio:

nb = read_nb(nbs[9])
L(nb['cells'])[1]

That's nice that we can get any cell, and get its info!

Jupyter Notebook Cells

Okay, let's grab the source of cells:

def get_source(cell): return cell['source']

cells = L(nb['cells']).map(get_source)
cells

Let's see if those are AttrDicts:

nb.cells[0]

Yes!

Let's use this nice AttrDict to get the source of a cell:

L(nb.cells)[1].source

def get_source(cell): return cell.source
cells = L(nb.cells).map(get_source)
cells

Yes, did it!

Jupyter Notebook Metadata

Sooo...besides cells there's metadata, right?

nb.metadata

Seems useful. Might be worth printing the Python version at least:

nb.metadata.language_info.version

I feel like I'd want to print the Python version for every notebook I'm publishing.

The version of each imported package would be nice too. Looks like that's beyond the scope of execnb most likely. Or is it?

CaptureShell

Looks like we can run a Jupyter notebook cell:

s = CaptureShell(mpl_format='retina')
s

s.run_cell('print("hi")')

Printing didn't have a result. How about a Python expression:

s.run_cell('1+1+1')

Ah, so we can see the result of evaluating it.

What about a Markdown cell?

s.run_cell('# Hi')

s.run('# Hi')

Looking at execnb.shell, I think it's just for Python code right now.

s.run("print(1)")

Rendering cell outputs

I'm offline and can't install dependencies, but I can see that render_outputs can render some outputs of executed cells like matplotlib plots.

What about a simple Python expression?

o = s.run("1+2+3")
o

render_outputs(o)

What about some FastHTML?

o = s.run("""from fasthtml.common import *
P("Hi")""")
o

render_outputs(o)

The HTML function from IPython.display looks useful here:

HTML(render_outputs(o))

Completions

SmartCompleter extends IPCompleter from IPython.core.completer. We can try instantiating one and seeing what it does:

cc = SmartCompleter(get_ipython())
cc

cc("pr")

This is quite interesting!

I'm currently offline and feel like I need to read the IPython docs and source to understand more.

Putting Pieces Together

Let's revisit a simple notebook and put these pieces together.

nb = read_nb(nbs[9])
cells = L(nb['cells'])
cells[0]

cells[0].source

Markdown(cells[0].source)

cells[1]

Markdown(cells[1].source)

cells[2]

HTML(cells[2].source)

s = CaptureShell(mpl_format='retina')
s

render_outputs(cells[2].outputs)

HTML(render_outputs(cells[2].outputs))

s.cell(cells[2])

cells[2].outputs

find_output(cells[2].outputs)['data']

out_exec(cells[2].outputs)

def get_type_and_source(cell): return cell.cell_type, cell.source

cells = L(nb.cells).map(get_type_and_source)
cells

def render_cell(c):
    if c.cell_type == 'markdown': return Markdown(c.source)
    # TODO: render both source and outputs for code cells
    elif c.cell_type == 'code': return HTML(c.source)

cells = L(nb.cells).map(render_cell)
cells

cells[0]

cells[1]

cells[5]

Reflect

I've studied the 2 notebooks in execnb: nbio and shell

I created examples to explore:

I reviewed the examples I created, putting them together using a larger portion of a sample notebook.

Next steps: