utils

We would like to add some cells programmatically. There seem to be some limitations given by the design of Jupyter; basically the file on disk, the kernel, and the frontend are different things. The following “payload” approach seems to be the only way to create a new cell without writing extensions for different frontends. However, this approach seems to be at risk of deprecation.

source

add_cell

def add_cell(
    content:str, single:bool=False, # If True, overwrites (updates) newly created cell when called multiple times.
):

Add a new cell below the calling cell once overall execution of the cell finishes.

Parameters

single : int If True, overwrites (updates) newly created cell when called multiple times.

add_cell("# Hi")

# Hi

# with single = True, the new cell will be updated (overwritten)
add_cell("# first", single=True)
add_cell("# second", single=True)

# second

source

add_cells

def add_cells(
    contents:list[str], # The contents to be placed in the new cells.
):

Adds several cells below the calling cells once excution of the current cell finishes

add_cells(["# Hi", "# there"])

# Hi

# there

add_cells([]) # This should not do anything

The calls to the payload manager are scheduled and executed in order after the cell execution finishes.

add_cell("# first")
add_cell("# second")

# second

# first

This means if you don’t reverse the desired output order, cells will be in the wrong order. The last cell needs to be added (scheduled) first. This makes streaming impossible - you need to know the end first. You can only post-process fully received output.

Let’s post-process markdown strings - add one cell per header (up to a certain level). This is helpful to split up long texts in small logical chunks, e.g., when importing a paper to Jupyter.

source

split_markdown

def split_markdown(
    md:str, max_level:int=6
)->list[str]:

Splits a markdown formatted string at headers

Let’s assign a long_string to demonstrate a use case:

long_string = """# Cell 1
This is the content for the first cell.

## Section 2

If `max_level` is larger than 1, this section will be its own cell.

# Final cell
A last top-level cell with `# some comment`.
"""

assert len(split_markdown(long_string, max_level=2)) == 3

The long document can then be added as individual cells like this:

add_cells(split_markdown(long_string, max_level=4))

Cell 1

This is the content for the first cell.

Section 2

If max_level is larger than 1, this section will be its own cell.

Final cell

A last top-level cell with # some comment.

The resulting cells are unfortunately code cells - you need to go manually through them to convert them to markdown (m) and execute (Shift + Enter, execute and go to next cell). I currently don’t have a good idea to make this automatic because as I understand this would only be possible from the frontend and therefore would require specialized code for each different frontend.

source

split_into_cells

def split_into_cells(
    s:str
)->tuple[list[str], list[str]]:

Take a markdown formatted text and split at triple backticks and insert references to cells