{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Progress Bars\n", "\n", "## Basic progress bars\n", "\n", "Progress bars are an excellent way to get an idea of how long a certain computation might take. Most of the methods responsible for computations or aggregations in Vaex support the display of progressbars. Displaying progress bars is as easy as:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "ExecuteTime": { "end_time": "2022-01-25T17:15:16.557923Z", "start_time": "2022-01-25T17:15:15.151448Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mean [########################################] 100.00% elapsed time : 0.09s = 0.0m = 0.0h\n", " " ] }, { "data": { "text/plain": [ "array(11.6269824)" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import vaex\n", "\n", "df = vaex.datasets.taxi()\n", "df.total_amount.mean(progress=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you are in the Jupyter notebook, you can pass `progress='widget'` to get a nicer looking progress bar, provided by [ipywidgets](https://ipywidgets.readthedocs.io/):" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "ExecuteTime": { "end_time": "2022-01-25T17:15:17.986939Z", "start_time": "2022-01-25T17:15:16.558697Z" } }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "84f530451b3f49dbb111239e51f5bd84", "version_major": 2, "version_minor": 0 }, "text/plain": [ "HBox(children=(FloatProgress(value=0.0, max=1.0), Label(value='In progress...')))" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "['CRD', 'CSH']" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.payment_type.unique(progress='widget')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Rich based progress bars\n", "\n", "Using [Rich](https://rich.readthedocs.io/) based progress bars we can take this idea to the next level. With Rich one gets to see a tree structure of progress bars that give the user an idea of what Vaex does internally, and how long each step takes. Each leaf in this tree is a `Task`, while the nodes are used to group tasks logically. For instance, in the following example the last node named 'mean' uses the mean aggregation, which creates two tasks: sum and count agregations." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "ExecuteTime": { "end_time": "2022-01-25T17:15:19.924928Z", "start_time": "2022-01-25T17:15:17.988220Z" } }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "604380654c764851853663bfa9356766", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "with vaex.progress.tree('rich', title=\"My Vaex computations\"):\n", " result_1 = df.groupby('passenger_count', agg='count')\n", " result_2 = df.groupby('vendor_id', agg=vaex.agg.sum('tip_amount')) \n", " result_3 = df.tip_amount.mean()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the last column (between brackets) we also see how many passes over the data Vaex had to do to compute all results. The last two tasks are done together in the 5th pass.\n", "\n", "If we want to do all computations in a single pass over the data for performance reason, we can use Vaex' async way, by adding the delayed argument (see [Async programming with Vaex](async.ipynb) for more details)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "ExecuteTime": { "end_time": "2022-01-25T17:15:21.498758Z", "start_time": "2022-01-25T17:15:19.925795Z" } }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9963b93951384e419a50e62688758e28", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "with vaex.progress.tree('rich', title=\"My Vaex computations\"):\n", " result_1 = df.groupby('passenger_count', agg='count', delay=True)\n", " result_2 = df.groupby('vendor_id', agg=vaex.agg.sum('tip_amount'), delay=True) \n", " result_3 = df.tip_amount.mean(delay=True)\n", " df.execute()\n", "result_1 = result_1.get()\n", "result_2 = result_2.get()\n", "result_3 = result_3.get()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that all computations are done in a single pass over the data, which is slightly faster in this case because we are not IO bound. On slower disks, or slower formats (e.g. parquet) this difference will be larger.\n", "\n", "Combining this with the [caching](caching.ipynb) feature, we can clearly see the effect on later calculations, and the efficiency of Vaex:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "ExecuteTime": { "end_time": "2022-01-25T17:15:23.336907Z", "start_time": "2022-01-25T17:15:21.499719Z" } }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "666c503cac7143e8add3b006f4effbe0", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "bb6618653d4a4f1f95d7ae4536873027", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Output()" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "vaex.cache.disk(clear=True) # turn on cache, and delete all cache entries\n", "\n", "with vaex.progress.tree('rich', title=\"Warm up cache\"):\n", " result_1 = df.groupby('passenger_count', agg='count', delay=True)\n", " result_2 = df.groupby('vendor_id', agg=vaex.agg.sum('tip_amount'), delay=True) \n", " df.execute()\n", "\n", "\n", "with vaex.progress.tree('rich', title=\"My Vaex computations\"):\n", " result_1 = df.groupby('passenger_count', agg='count', delay=True)\n", " result_2 = df.groupby('vendor_id', agg=vaex.agg.sum('tip_amount'), delay=True) \n", " result_3 = df.tip_amount.mean(delay=True)\n", " df.execute()\n", "vaex.cache.off();" ] } ], "metadata": { "interpreter": { "hash": "2b337e1aa502f5cea9a92c761ad75d3ab5045107ee3446fdbe7f873d4f1936e7" }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": { "06375b830af74c4997b21febdaf77735": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "description_width": "" } }, "355b8d9d00d34629b826543dde2428a5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "description_width": "" } }, "5638c565df0e48e1b86d41c5d3748ed2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "5aa60c487c5e417fba0ee12787d642a7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "layout": "IPY_MODEL_5638c565df0e48e1b86d41c5d3748ed2", "max": 1, "style": "IPY_MODEL_06375b830af74c4997b21febdaf77735", "value": 1 } }, "604380654c764851853663bfa9356766": { "model_module": "@jupyter-widgets/output", "model_module_version": "1.0.0", "model_name": "OutputModel", "state": { "layout": "IPY_MODEL_818c8a2873054c8db46c948945c8903e", "outputs": [ { "data": { "text/html": "
  My Vaex computations                          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.89s     \n├──   groupby                                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.45s     \n│   ├──   groupers                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.30s     \n│   │   └──   grouper: 'passenger_count'        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.30s     \n│   │       └──   set for passenger_count       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.30s[1]  \n│   └──   aggregators                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.14s     \n│       └──   vaex.agg.count('*')               ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.14s[2]  \n├──   groupby                                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.32s     \n│   ├──   groupers                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.17s     \n│   │   └──   grouper: 'vendor_id'              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.17s     \n│   │       └──   set for vendor_id             ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.17s[3]  \n│   └──   aggregators                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.14s     \n│       ├──   vaex.agg.sum('tip_amount')        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.14s[4]  \n│       └──   vaex.agg.count('*')               ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.14s[4]  \n└──   mean                                      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.07s     \n    └──   vaex.agg.mean('tip_amount')           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.07s     \n        ├──   vaex.agg.sum('tip_amount')        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.07s[5]  \n        └──   vaex.agg.count('tip_amount')      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.07s[5]  \n
\n", "text/plain": " \u001b[31mMy Vaex computations \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.89s \u001b[0m \n├── \u001b[31mgroupby \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.45s \u001b[0m \n│ ├── \u001b[31mgroupers \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.30s \u001b[0m \n│ │ └── \u001b[31mgrouper: 'passenger_count' \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.30s \u001b[0m \n│ │ └── \u001b[31mset for passenger_count \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.30s[1]\u001b[0m \n│ └── \u001b[31maggregators \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.14s \u001b[0m \n│ └── \u001b[31mvaex.agg.count('*') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.14s[2]\u001b[0m \n├── \u001b[31mgroupby \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.32s \u001b[0m \n│ ├── \u001b[31mgroupers \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.17s \u001b[0m \n│ │ └── \u001b[31mgrouper: 'vendor_id' \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.17s \u001b[0m \n│ │ └── \u001b[31mset for vendor_id \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.17s[3]\u001b[0m \n│ └── \u001b[31maggregators \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.14s \u001b[0m \n│ ├── \u001b[31mvaex.agg.sum('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.14s[4]\u001b[0m \n│ └── \u001b[31mvaex.agg.count('*') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.14s[4]\u001b[0m \n└── \u001b[31mmean \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.07s \u001b[0m \n └── \u001b[31mvaex.agg.mean('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.07s \u001b[0m \n ├── \u001b[31mvaex.agg.sum('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.07s[5]\u001b[0m \n └── \u001b[31mvaex.agg.count('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.07s[5]\u001b[0m \n" }, "metadata": {}, "output_type": "display_data" } ] } }, "663e636d231948178dadb92bf1e5a019": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "666c503cac7143e8add3b006f4effbe0": { "model_module": "@jupyter-widgets/output", "model_module_version": "1.0.0", "model_name": "OutputModel", "state": { "layout": "IPY_MODEL_a798e251c59b4f2baaea56b10fcb2f22", "outputs": [ { "data": { "text/html": "
  Warm up cache                                 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.56s     \n├──   groupby                                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.56s     \n│   ├──   groupers                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.34s     \n│   │   └──   grouper: 'passenger_count'        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.34s     \n│   │       └──   set for passenger_count       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.34s[1]  \n│   └──   aggregators                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.21s     \n│       └──   vaex.agg.count('*')               ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.20s[2]  \n└──   groupby                                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.56s     \n    ├──   groupers                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.34s     \n    │   └──   grouper: 'vendor_id'              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.34s     \n    │       └──   set for vendor_id             ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.34s[1]  \n    └──   aggregators                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.21s     \n        ├──   vaex.agg.sum('tip_amount')        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.20s[2]  \n        └──   vaex.agg.count('*')               ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.20s[2]  \n
\n", "text/plain": " \u001b[31mWarm up cache \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.56s \u001b[0m \n├── \u001b[31mgroupby \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.56s \u001b[0m \n│ ├── \u001b[31mgroupers \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.34s \u001b[0m \n│ │ └── \u001b[31mgrouper: 'passenger_count' \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.34s \u001b[0m \n│ │ └── \u001b[31mset for passenger_count \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.34s[1]\u001b[0m \n│ └── \u001b[31maggregators \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.21s \u001b[0m \n│ └── \u001b[31mvaex.agg.count('*') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.20s[2]\u001b[0m \n└── \u001b[31mgroupby \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.56s \u001b[0m \n ├── \u001b[31mgroupers \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.34s \u001b[0m \n │ └── \u001b[31mgrouper: 'vendor_id' \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.34s \u001b[0m \n │ └── \u001b[31mset for vendor_id \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.34s[1]\u001b[0m \n └── \u001b[31maggregators \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.21s \u001b[0m \n ├── \u001b[31mvaex.agg.sum('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.20s[2]\u001b[0m \n └── \u001b[31mvaex.agg.count('*') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.20s[2]\u001b[0m \n" }, "metadata": {}, "output_type": "display_data" } ] } }, "753ca65e55fb4352a73041f42a4ad9b3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "LabelModel", "state": { "layout": "IPY_MODEL_b631026f9a1a4f6ba7baa8b61483be33", "style": "IPY_MODEL_355b8d9d00d34629b826543dde2428a5", "value": " 100.00% elapsed time : 1.39s = 0.0m = 0.0h cpu: 552%" } }, "818c8a2873054c8db46c948945c8903e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "84f530451b3f49dbb111239e51f5bd84": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "children": [ "IPY_MODEL_5aa60c487c5e417fba0ee12787d642a7", "IPY_MODEL_753ca65e55fb4352a73041f42a4ad9b3" ], "layout": "IPY_MODEL_b9b4a9bbdb5a4b7e848417af9248a866" } }, "9963b93951384e419a50e62688758e28": { "model_module": "@jupyter-widgets/output", "model_module_version": "1.0.0", "model_name": "OutputModel", "state": { "layout": "IPY_MODEL_663e636d231948178dadb92bf1e5a019", "outputs": [ { "data": { "text/html": "
  My Vaex computations                          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.52s     \n├──   groupby                                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.52s     \n│   ├──   groupers                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.29s     \n│   │   └──   grouper: 'passenger_count'        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.29s     \n│   │       └──   set for passenger_count       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.28s[1]  \n│   └──   aggregators                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.23s     \n│       └──   vaex.agg.count('*')               ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.22s[2]  \n├──   groupby                                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.52s     \n│   ├──   groupers                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.28s     \n│   │   └──   grouper: 'vendor_id'              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.28s     \n│   │       └──   set for vendor_id             ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.28s[1]  \n│   └──   aggregators                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.23s     \n│       ├──   vaex.agg.sum('tip_amount')        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.22s[2]  \n│       └──   vaex.agg.count('*')               ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.22s[2]  \n└──   mean                                      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.28s     \n    └──   vaex.agg.mean('tip_amount')           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.28s     \n        ├──   vaex.agg.sum('tip_amount')        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.28s[1]  \n        └──   vaex.agg.count('tip_amount')      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 01.28s[1]  \n
\n", "text/plain": " \u001b[31mMy Vaex computations \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.52s \u001b[0m \n├── \u001b[31mgroupby \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.52s \u001b[0m \n│ ├── \u001b[31mgroupers \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.29s \u001b[0m \n│ │ └── \u001b[31mgrouper: 'passenger_count' \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.29s \u001b[0m \n│ │ └── \u001b[31mset for passenger_count \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.28s[1]\u001b[0m \n│ └── \u001b[31maggregators \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.23s \u001b[0m \n│ └── \u001b[31mvaex.agg.count('*') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.22s[2]\u001b[0m \n├── \u001b[31mgroupby \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.52s \u001b[0m \n│ ├── \u001b[31mgroupers \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.28s \u001b[0m \n│ │ └── \u001b[31mgrouper: 'vendor_id' \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.28s \u001b[0m \n│ │ └── \u001b[31mset for vendor_id \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.28s[1]\u001b[0m \n│ └── \u001b[31maggregators \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.23s \u001b[0m \n│ ├── \u001b[31mvaex.agg.sum('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.22s[2]\u001b[0m \n│ └── \u001b[31mvaex.agg.count('*') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.22s[2]\u001b[0m \n└── \u001b[31mmean \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.28s \u001b[0m \n └── \u001b[31mvaex.agg.mean('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.28s \u001b[0m \n ├── \u001b[31mvaex.agg.sum('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.28s[1]\u001b[0m \n └── \u001b[31mvaex.agg.count('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m01.28s[1]\u001b[0m \n" }, "metadata": {}, "output_type": "display_data" } ] } }, "a798e251c59b4f2baaea56b10fcb2f22": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "b631026f9a1a4f6ba7baa8b61483be33": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "b9b4a9bbdb5a4b7e848417af9248a866": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "bb6618653d4a4f1f95d7ae4536873027": { "model_module": "@jupyter-widgets/output", "model_module_version": "1.0.0", "model_name": "OutputModel", "state": { "layout": "IPY_MODEL_cc2dac2271a846ea9edc13c5e72bbc2e", "outputs": [ { "data": { "text/html": "
  My Vaex computations                          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.14s     \n├──  groupby                                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.16s     \n│   ├──   groupers                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.00s     \n│   │   └──   grouper: 'passenger_count'        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.00s     \n│   │       └──   set for passenger_count       ━━━━━━━━━━━━━━━━━━━ 100% 00.00s    from cache\n│   └──   aggregators                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.00s     \n│       └──   vaex.agg.count('*')               ━━━━━━━━━━━━━━━━━━━ 100% 00.00s    from cache\n├──  groupby                                   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.14s     \n│   ├──   groupers                              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.00s     \n│   │   └──   grouper: 'vendor_id'              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.00s     \n│   │       └──   set for vendor_id             ━━━━━━━━━━━━━━━━━━━ 100% 00.00s    from cache\n│   └──  aggregators                           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.13s     \n│       ├──   vaex.agg.sum('tip_amount')        ━━━━━━━━━━━━━━━━━━━ 100% 00.00s    from cache\n│       └──   vaex.agg.count('*')               ━━━━━━━━━━━━━━━━━━━ 100% 00.00s    from cache\n└──   mean                                      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.07s     \n    └──   vaex.agg.mean('tip_amount')           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.07s     \n        ├──   vaex.agg.sum('tip_amount')        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.07s[1]  \n        └──   vaex.agg.count('tip_amount')      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 00.07s[1]  \n
\n", "text/plain": " \u001b[31mMy Vaex computations \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.14s \u001b[0m \n├── \u001b[32m⠙\u001b[0m \u001b[31mgroupby \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.16s \u001b[0m \n│ ├── \u001b[31mgroupers \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \n│ │ └── \u001b[31mgrouper: 'passenger_count' \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \n│ │ └── \u001b[31mset for passenger_count \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \u001b[31mfrom cache\u001b[0m\n│ └── \u001b[31maggregators \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \n│ └── \u001b[31mvaex.agg.count('*') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \u001b[31mfrom cache\u001b[0m\n├── \u001b[32m⠙\u001b[0m \u001b[31mgroupby \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.14s \u001b[0m \n│ ├── \u001b[31mgroupers \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \n│ │ └── \u001b[31mgrouper: 'vendor_id' \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \n│ │ └── \u001b[31mset for vendor_id \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \u001b[31mfrom cache\u001b[0m\n│ └── \u001b[32m⠙\u001b[0m \u001b[31maggregators \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.13s \u001b[0m \n│ ├── \u001b[31mvaex.agg.sum('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \u001b[31mfrom cache\u001b[0m\n│ └── \u001b[31mvaex.agg.count('*') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.00s \u001b[0m \u001b[31mfrom cache\u001b[0m\n└── \u001b[31mmean \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.07s \u001b[0m \n └── \u001b[31mvaex.agg.mean('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.07s \u001b[0m \n ├── \u001b[31mvaex.agg.sum('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.07s[1]\u001b[0m \n └── \u001b[31mvaex.agg.count('tip_amount') \u001b[0m \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[33m00.07s[1]\u001b[0m \n" }, "metadata": {}, "output_type": "display_data" } ] } }, "cc2dac2271a846ea9edc13c5e72bbc2e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} } }, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 2 }