Execution API¶

nbexec adds an /api/executions resource to the Jupyter notebook RESTful web API.

Execution Model¶

nbexec creates an ‘Execution’ object on executing a notebook. ‘Execution Model’ is a JSON representation of Execution which has the following properties. The properties marked with an asterisk (*) are always included.

Name	Type	Description
exec_id *	str	Execution ID (UUID).
path *	str	Notebook file path relative to the Jupyter notebook’s root directory.
params *	object	Runtime parameters given from an API request. The values are stored as string instead of defined types in a notebook.
output_path	str	Output file path. It is `null` if `output_path` parameter was not specified and the execution is not finished yet. It is also `null` if the execution is failed to start (e.g. the notebook is broken, the file is not a notebook).
overwrite *	bool	Whether to overwrite the existing output file.
jupyter_kernel	str	Jupyter kernel name.
cell_timeout	int	Cell timeout in seconds.
status *	str	Execution status. ‘initializing’, ‘executing’, ‘completed’ or ‘error: <msg>’
progress	str	Execution progress which indicates the current executing cell number and the total number of cells (e.g. ‘5/15’). It is `null` if the execution is not started yet.
last_cell_source	str	The source code of the last started cell. It is `null` if any cell has not yet been execute.
started_at	float	Timestamp of the started time. It is `null` if the execution is not started yet.
completed_at	float	Timestamp of the completed time. It is `null` if the execution is not completed yet.

Execution Life Cycle¶

Executions are stored in a in-memory DB by Jupyter notebook server same as notebook sessions. An execution object is created just before the notebook execution and remains in the DB until it is manually deleted. If you create many executions, Jupyter notebook server consumes much memory. It is recommended to delete unused executions periodically or just after they finished.

When Jupyter notebook server shuts down, execution data will be lost. They are not persisted beyond Jupyter notebook server’s life time.

API Spec¶

Common Parameters¶

All API requests must have token. See Authentication.

Sets all parameters into a request body using URL-encoded form data (application/x-www-form-urlencoded) if the HTTP method is POST, PUT or PATCH; otherwise sets all parameters as a query string into the URL.

Error Response¶

The error response will depend on the following HTTP status codes:

HTTP status codes¶

Status code	Description
400 Bad Request	The request was not processed, as the server could not understand what the client is asking for.
401 Unauthorized	The request lacks valid credentials and it should re-request with the required credentials.
404 Not Found	The requested resource was not found.
500 Internal Server Error	The request is valid, but the server could not fulfill it due to some unexpected condition.

The error message is nothing more than reference information for the error code.

GET /api/executions¶

Gets the list of execution models.

Response body (JSON)¶

Key	Type	Description
executions	list	The list of execution models.

HTTP status codes¶

Status code	Description
200 OK	The request has succeeded. The list of execution models is in the request body.
401 Unauthorized	Unauthorized request.

POST /api/executions¶

Executes the notebook with given parameters with creating an Execution object. The working directory of the execution is same as the parent directory of the notebook.

Parameters¶

The parameters except below are passed as notebook parameters. The parameters marked with an asterisk (*) are required.

name	Description
notebook *	Notebook file path to be executed. The path is relative to the Jupyter notebook’s root directory.
output_path	Output notebook file path. The path is relative to the Jupyter notebook’s root directory. If it is not specified, it is going to be `<path>-Executed1.ipynb`.
overwrite	Whether to overwrite the output notebook file. `output_path` must be set if it is `true`. If it is `false`, the file name is incremented such as `<path>-Executed2.ipynb`.
jupyter_kernel	Jupyter kernel name to execute the notebook. Only IPython kernel is available. If it is not specified, the default kernel of the Jupyter notebook is used.
cell_timeout	Timeout for each cell in seconds. When a cell takes more time than it, the whole execution will be terminated immediately. If it is not specified, cells are never timed out.

Response bosy (JSON)¶

Key	Type	Description
event	str	Name of the event
timestamp	float	Time when the event happened
execution	object	An execution model.

HTTP status codes¶

Status code	Description
202 Accepted	The execution is started. It sending progress payloads using chunked transfer encoding while the execution is completed if `X-Response-Encoding: chunked` is set; otherwise only `notebook_start` event is sent and the execution is run asynchronously.
400 Bad Request	Invalid parameters are specified.
401 Unauthorized	Unauthorized request.
404 Not Found	The notebook file is not found.

Note

It launches a new process to run the notebook on a Jupyter kernel for each call.
It uses 5 random ports which are selected from unused ports to launch Jupyter kernel.

DELETE /api/executions¶

Deletes all executions with shutting down the IPython kernels. This API requires no parameter and returns an empty response body.

HTTP status codes¶

Status code	Description
202 Accepted	The request has been accepted. It waits deletion is completed if `X-Response-Encoding: chunked` is set; otherwise the deletion is run asynchronously.
401 Unauthorized	Unauthorized request.

GET /api/executions/<exec_id>¶

Gets the execution model of the given exec_id.

Response bosy (JSON)¶

Key	Type	Description
execution	object	An execution model.

HTTP status codes¶

Status code	Description
200 OK	The request has succeeded. The execution model is in the request body.
401 Unauthorized	Unauthorized request.

POST /api/executions/<exec_id>¶

Executes an action to the given execution. The only available action is shutdown.

Parameters¶

Parameters marked with an asterisk (*) are always included.

name	Description
action *	The action to be executed. The only available action is `shutdown` which shuts down the IPython kernel which is used by the execution.

Response bosy (JSON)¶

This API returns an empty response body. If X-Response-Encoding: chunked is set in the request header, this API returns the following response body.

Key	Type	Description
execution	object	An execution model.

HTTP status codes¶

Status code	Description
202 Accepted	The request has been accepted. It waits the operation is completed if `X-Response-Encoding: chunked` is set; otherwise the operation is run asynchronously.
400 Bad Request	Invalid parameters are specified.
401 Unauthorized	Unauthorized request.

DELETE /api/executions/<exec_id>¶

Deletes the given execution with shutting down the IPython kernel if it is still alive. This API requires no parameter and returns an empty response body.

HTTP status codes¶

Status code	Description
202 Accepted	The request has been accepted. It waits deletion is completed if `X-Response-Encoding: chunked` is set; otherwise the deletion is run asynchronously.
401 Unauthorized	Unauthorized request.

Progress Payload¶

All payloads has this common structure:

{
  "event": "<event_name>",
  "timestamp": <timestamp>,
  ...
}

event is a name of the event, and timestamp is a time when the event happened. The timestamp value is a float that datetime.datetime.timestamp() returns. The top-level object can have other key-values, which depend on the event type.

Events¶

notebook_start: the notebook started
notebook_complete: the notebook completes all cells
start: A cell started
end: A cell ended
notebook_error: the notebook ended with error

Notebook Events¶

notebook_start and notebook_complete events have execution which has an Execution Model as its value.

Cell Events¶

start and end events have progress and cell. progress indicates the current notebook progress in format <current_cell_num>/<total_cell_num>. cell is the same object which exists in .ipynb file.

Examples¶

notebook_start:

{
  "event": "notebook_start",
  "timestamp": 1510621481.601824,
  "execution": {
    "exec_id": "f5022ad6-ad17-453f-9d0a-7d7f5bc019c9",
    "path": "/home/user/examples/Example1.ipynb",
    "params": {},
    "output_path": null,
    "overwrite": false,
    "jupyter_kernel": null,
    "cell_timeout": null,
    "status": "executing",
    "progress": null,
    "last_cell_source": null,
    "started_at": 1510621481.601824,
    "completed_at": null
  }
}

notebook_complete:

{
  "event": "notebook_complete",
  "timestamp": 1510621487.121837,
  "execution": {
    "exec_id": "f5022ad6-ad17-453f-9d0a-7d7f5bc019c9",
    "path": "/home/user/examples/Example1.ipynb",
    "params": {},
    "output_path": "/home/user/examples/Example1-Executed1.ipynb",
    "overwrite": false,
    "jupyter_kernel": null,
    "cell_timeout": null,
    "status": "completed",
    "progress": "10/10",
    "last_cell_source": "1 + 10",
    "started_at": 1510621481.601824,
    "completed_at": 1510621487.121837
  }
}

start:

{
  "event": "start",
  "timestamp": 1510621482.63849,
  "progress": "1/10",
  "cell": {
    "outputs": [],
    "cell_type": "code",
    "metadata": {
      "nbexec": {
        "start_time": "2017-11-14T01:04:42.638490+00:00"
      },
      "collapsed": true
    },
    "execution_count": 1,
    "source": "1 + 2"
  }
}

end:

{
  "event": "end",
  "timestamp": 1510621482.650049,
  "progress": "1/10",
  "cell": {
    "outputs": [
      {
        "output_type": "execute_result",
        "data": {
          "text/plain": "3"
        },
        "execution_count": 1,
        "metadata": {}
      }
    ],
    "cell_type": "code",
    "metadata": {
      "nbexec": {
        "end_time": "2017-11-14T01:04:42.650049+00:00",
        "start_time": "2017-11-14T01:04:42.638490+00:00",
        "duration": 0.011559
      },
      "collapsed": true
    },
    "execution_count": 1,
    "source": "1 + 2"
  }
}

notebook_error:

{
  "event": "notebook_error",
  "timestamp": 1518170320.507459,
  "output_path": "/home/user/examples/Error-Executed1.ipynb",
  "error": "An error occurred while executing the following cell:\n------------------\nraise ValueError()\n------------------\n\nValueError: \n"
}

Output File¶

The output file name of ‘Example.ipynb’ is by default ‘Example-Executed1.ipynb’ and the last number is incremented if the same file name already exists.

Authentication¶

The execution API always requires token parameter (Jupyter token) and does not use Cookies.

Limitations¶

POST /api/executions synchronously and GET /api/executions might rarely return the following error:
```
{
  "error": "Kernel died before replying to kernel_info",
  "event": "notebook_error",
  "exec_id": "18e3be0a-6300-4204-af2c-fb9c129c7fa9",
  "output_path": "sample-Executed567.ipynb",
  "timestamp": 1518570812.971526
}
```
It causes the Jupyter kernel failed to open unused ports on startup because they are used by another process before Jupyter kernel open them. Please call POST /api/executions to execute the notebook again if it is caused. You may limit the concurrent calls of POST /api/executions because the too many concurrent calls of POST /api/executions increase the possibility of this error. See also this issue for more details.