Computer Vision, NLP, and Gaming in the Browser

Unleash the power of Python Panel, Pyodide, and PyScript

By Sophia Yang and Marc Skov Madsen

We now live in an era where we can run Python in the browser, with the power of WebAssembly, Pyodide, and PyScript! What does that mean for data practitioners and developers? It means that we can run computer vision tasks, NLP tasks, and even develop games entirely in Python and run your Python code directly in the browser. Everything happens in your browser!

This article will show you three simple examples demonstrating the potential of running a Python computer vision app, a text analysis app, and a simple Trivia game in the browser.

All three examples in this article start with a Python Panel dashboard. Panel is the dashboarding library from the open-source HoloViz ecosystem. To learn about how to make a Panel dashboard in Python, check out our previous blog post on the three main ways to build a Panel dashboard and how to deploy a Panel visualization dashboard to Github pages.

Once you have a Panel app app.py, with one line of code using panel convert, you can convert your Panel application to WebAssembly (many thanks to Philipp Rudiger for this feature):

panel convert app.py — to pyodide-worker — out docs

Panel convert can either use pyodide or pyscript as the backend, but a pyodide worker is currently the faster option. Check out the panel documentation to learn about different parameters you can choose.

Try the app yourself: https://awesome-panel.github.io/examples/videostream-interface/app.html

Code: https://github.com/awesome-panel/examples/blob/main/src/videostream-interface/app.py

Our first example shows a computer vision app to display a video from a webcam local stream and perform real-time video transformation and face detection. There are two main components of the app:

• pn.widgets.VideoStream is a Panel widget that allows us to interact with the webcam stream.
• Scikit-image is an image-processing library in Python, which handles all transformations and face detections.

In just a few hundred lines of Python, you can easily build an app that does sophisticated image processing like this, building on Python’s extensive ecosystem of open-source libraries.

Try the app yourself: https://sophiamyang.github.io/panel_simple_text_analysis/text_analysis.html

Code: https://github.com/sophiamyang/panel_simple_text_analysis/blob/main/text_analysis.ipynb

Our second example shows a simple text analysis. You can either type in text in the input text box or upload a file and then you will see the sentiment analysis, N-gram analysis, and topic modeling result of this text:

• The sentiment analysis uses TextBlob and returns polarity and subjectivity. Polarity ranges from -1 to 1, with -1 being negative and 1 being positive. The TextBlob can also use the subjectivity function to calculate subjectivity, which ranges from 0 to 1, with 0 being objective and 1 being subjective.
• N-gram analyses are often used to see which words often show up together. We often like to investigate combinations of two words or three words, i.e., Bigrams/Trigrams.
• There are many ways to conduct topic modeling. In this example, we use the Non-Negative Matrix Factorization (NMF) method.

To learn more about basic text analysis in Python, check out our previous blog post on this topic.

Try the app yourself: https://sophiamyang.github.io/panel_trivia_game/trivia.html

Code: https://github.com/sophiamyang/panel_trivia_game/blob/main/trivia.ipynb

Can you make a game and run it in the browser? Yes, absolutely! Your Panel app does not need to be data related at all.

In this example, we’ve built a very simple Trivia game app. We define the number of questions, difficulty level, and category of the Trivia game, and then we will see a list of questions for us to decide whether each is true or false. Two things to call out for this app:

• We use requests to fetch Trivia game questions from the Open Triva Database API.
• Similar to methods like SetOnClickListener in Java and similar methods in other languages, with Panel, we can pass a Python function on_click to determine what to execute when there is a click.

These three apps are just a start to demonstrate the potential of what we can do with Panel and the browser, with no server required! With a bit more effort, you can make a lot more complicated Panel applications and run them in the browser. Please feel free to share your applications at https://discourse.holoviz.org/ and we look forward to seeing what you create.

Thank you Jim Bednar and Phillipp Rudiger for your guidance and feedback!

. . .

By Sophia Yang and Marc Skov Madsen on October 26, 2022.

Sophia Yang is a Senior Data Scientist at Anaconda. Connect with me on LinkedIn, Twitter, and YouTube, and join the DS/ML Book Club ❤️

Computer Vision, NLP, and Gaming in the Browser Republished from Source https://towardsdatascience.com/computer-vision-nlp-and-gaming-in-the-browser-8e5fcf895a14?source=rss—-7f60cf5620c9—4 via https://towardsdatascience.com/feed

<!–

–>