Enterprise Logging

Connect Mito to your internal logging servers so you can track Python retention at your firm.

Collecting Mito Enterprise Logs requires a Mito Enterprise License.

Why is Mito Enterprise Logging Useful?

Python adoption is often a black box at large enterprises. You might know who has access to Python or even how often they log in to a Python environment like JupyterHub, but you probably don't know for which users their Python code is delivering business value.

Mito Enterprise Logging is designed to help you gain visibility into Python retention at your firm. The logs are designed to help you calculate metrics like:

• Monthly Active Users of Mito

• The top 100 most active Mito power users at your firm

• The teams that have not yet adopted Python and require additional support

• The most commonly used Mito features at your firm

• The top 10 most common errors that users were blocked by

Metrics like these open the black box of Python adoption at your firm and make it easy to evaluate and communicate the success of Python automations.

Collecting Mito Enterprise Logs

Collecting Mito Enterprise Logs requires a Mito Enterprise License.

Setting up your log server

Mito is agnostic to the logging infrastructure that you use. Common log monitoring platforms include Datadog, Mixpanel, and Amplitude.

To get the most out of the logs that Mito generates, it may be useful to supplement the logs Mito generates with additional information about each user. For example: username, operating system, browser, environment, etc.

Connecting Mito to your log server

To collect logs, you must set the following environment variables in your user's Python environment:

MITO_CONFIG_VERSION='2'
MITO_CONFIG_LOG_SERVER_URL='https://url/of/log/server'
MITO_CONFIG_LOG_SERVER_BATCH_INTERVAL='<number of seconds between log uploads>'

Once configured, Mito will upload logs to the server:

1. In a batched manner. This reduces the burden on your log server.

2. The logs will upload at fixed time intervals, specified by MITO_CONFIG_LOG_SERVER_BATCH_INTERVAL. If no batch interval is provided, logs will be uploaded every 10 seconds by default.

3. From a separate thread than the main Mito processing thread. This ensures that log collection has a minimal effect on your users' experience.

4. If log upload fails:

   1. No logs will be lost. Logs will be saved and tried again on the next upload attempt.

   2. An exponential backoff strategy is used, so upload will be retried with double the batch interval. This exponential backoff strategy avoids overburdening the server.

Together, the above mean that logs may sometimes be dropped, especially if the Python process Mito is running in is terminated unexpectedly.

Logs Generated by Mito Enterprise

Mito Usage Events

Log Event	Description
mitosheet_rendered	A new or existing Mito spreadsheet was created

Mito Analysis Events

Data Import and Export Events

Log Event	Description
excel_import_edit	At least one sheet from an Excel workbook was imported
excel_range_import_edit	A range from an Excel worksheet was imported using either dynamic or static range detection
simple_import_edit	A CSV file was imported
dataframe_import_edit	A dataframe that was defined in the Jupyter notebook was imported using the Import Dataframe Taskpane
export_to_file_edit	Data in Mito was exported to a CSV or Excel file

Data Transformation Events

Log Event	Description
pivot_edit	A pivot table was created or updated
filter_column_edit	A column filter was applied or updated
sort_edit	The dataframe was sorted by a column
change_column_dtype_edit	A column's data type was changed
merge_edit	Two dataframes were merged together or an existing merge was updated
concat_edit	Dataframes were vertically concatonated on top of eachother to creatre a new dataframe
delete_column_edit	A column(s) were deleted
rename_column_edit	A column was renamed
add_column_edit	A column was added to the dataframe
set_column_formula_edit	A formula was created or updated
reorder_column_edit	A column's order in the dataframe was changed
fill_na_edit	NaN values were filled using the FillNaN Taskpane
delete_row_edit	A row(s) were deleted
drop_duplicates_edit	Duplicate values were removed from the dataframe using the Drop Duplicates Taskpane
split_text_to_columns_edit	A column was split on a delimiter into multiple columns using the Split Text to Columns Taskpane
promote_row_to_header_edit	A row was promoted to the header row
melt_edit	A dataframe was melted (unpivoted)
reset_index_edit	The dataframe's indexes were reset to the standard 0, 1, ... N
transpose_edit	A dataframe was transposed
dataframe_delete_edit	A dataframe was deleted
dataframe_duplicate_edit	A dataframes was duplicated
dataframe_rename_edit	A dataframe was renamed

Formatting Events

Log Event	Description
change_column_format_edit	A column had its format changed. Ie: From plain text to accounting format
set_dataframe_format_edit	A conditional formatting or dataframe color scheme was updated.

Graphing Events

Log Event	Description
graph_edit	A graph was created or an existing graph had it's configuration updated. For example, a bar chart was changed to a line chart.
graph_delete_edit	A graph was deleted
graph_rename_edit	A graph was renamed

Errors

Log Event	Description
error	All errors in Mito generate an error log event that contains useful information like the error traceback needed to identify and correct a bug. Use the log param params_failed_log_event to identify which event caused the error. For easy triaging, errors are tagged with an error_severity_code.
frontend_render_failed	The Mito Spreadsheet has completely crashed. This is the most severe error.

Error Severity Codes

Error Category	Error Severity Code
Likely just a warning	0
Likely user error	10
Likely inconsequential error	11
Likely Mito Bug	20
Unable to import Data	21
Unable to replay analysis	22
Mito Crashed	50
Misc. (unexpected errors)	-1

Example Logs

To help you understand the logs that Mito generates, below is a video of a short Mito session accompanied with the logs generated by that session. In this session the user:

1. Renders a Mito spreadsheet

2. Adds a column to the end of the dataframe

3. Renames the new column to New Column

4. Writes a spreadsheet formula, =LEFT(Last Name0, 2) to get the first two letters from the Last Name column

5. Generates an error by attempting to create a duplicate column header

[
    {
        "event": "mitosheet_rendered",
        "params_user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2.1 Safari/605.1.15", 
        "version_python": "3.9.6", 
        "version_pandas": "2.1.2", 
        "version_mito": "0.3.131", 
        "timestamp_gmt": "2024-01-25T23:07:59Z"
    }, 
    {
        "event": "add_column_edit",
        "params_sheet_index": 0, 
        "params_column_header": "childworldfall", 
        "params_column_header_index": 5, 
        "params_public_interface_version": 3, 
        "version_python": "3.9.6", 
        "version_pandas": "2.1.2", 
        "version_mito": "0.3.131", 
        "timestamp_gmt": "2024-01-25T23:08:03Z" 
    }, 
    {
        "event": "rename_column_edit",
        "params_sheet_index": 0, 
        "params_column_id": "childworldfall", 
        "params_new_column_header": "personpersonhat", 
        "params_public_interface_version": 3, 
        "version_python": "3.9.6", 
        "version_pandas": "2.1.2", 
        "version_mito": "0.3.131", 
        "timestamp_gmt": "2024-01-25T23:08:07Z", 
    }, 
    {
        "event": "set_column_formula_edit",
        "params_sheet_index": 0, 
        "params_column_id": "childworldfall", 
        "params_formula_label": "workfallpoint", 
        "params_new_formula": "=LEFT(companypartcaseI[workfallpoint], 2)", 
        "params_index_labels_formula_is_applied_to": {"type": "timepartcase"}, 
        "params_cell_editor_location": "cell", 
        "params_public_interface_version": 3, 
        "version_python": "3.9.6", 
        "version_pandas": "2.1.2", 
        "version_mito": "0.3.131", 
        "timestamp_gmt": "2024-01-25T23:08:14Z"
    }, 
    {
        "event": "error"
        "error_severity_code": "10",
        "params_failed_log_event": "rename_column_edit_failed",
        "params_sheet_index": 0, 
        "params_column_id": "womancasepoint", 
        "params_new_column_header": "timedogcase", 
        "params_public_interface_version": 3, 
        "error_traceback": [
            "Traceback (most recent call last):", "  File 'mito_backend.py', line 253, in receive_message", 
            "    self.handle_edit_event(event)", 
            "    File 'mito_backend.py', line 178, in handle_edit_event", 
            "    self.steps_manager.handle_edit_event(event)", 
            "    File 'steps_manager.py', line 523, in handle_edit_event", 
            "    self.execute_and_update_steps(new_steps)", 
            "    File 'steps_manager.py', line 745, in execute_and_update_steps", 
            "    final_steps = execute_step_list_from_index(", "  File 'steps_manager.py', line 101, in execute_step_list_from_index", 
            "    new_step.set_prev_state_and_execute(last_valid_step.final_defined_state, non_skipped_steps)", 
            "    File 'step.py', line 154, in set_prev_state_and_execute", 
            "    post_state_and_execution_data = self.step_performer.execute(new_prev_state, params)", 
            "    File 'rename_column.py', line 42, in execute", 
            "    raise make_column_exists_error(new_column_header)", 
            "    mitosheet.errors.MitoError: (column_exists_error, Column Already Exists, Sorry, a column already exists with the name application sign date. Try picking a different name!)"
        ], 
        "error_traceback_last_line": "mitosheet.errors.MitoError: (column_exists_error, Column Already Exists, Sorry, a column already exists with the name application sign date. Try picking a different name!\)", 
        "version_python": "3.9.6", 
        "version_pandas": "2.1.2", 
        "version_mito": "0.3.131", 
        "timestamp_gmt": "2024-01-25T23:08:20Z", 
    }
]

Debugging your logging configuration

If Mito is unable to upload logs, it will generate useful errors.

To see the logs in JupyterLab, click on View > Show Log Console , and inside of the log console, change the Log Level from Warning to Debug.

For example, with the following invalid enterprise logging configuration, you should see the following error messages:

import os 
os.environ['MITO_CONFIG_VERSION'] = '2'
os.environ['MITO_CONFIG_LOG_SERVER_URL'] = 'https://invalid-url'