First Date with Flask

These past few days, I’ve been taking a hiatus from the spree of neural networks and machine learning to explore an entirely separate realm of technology: web development. Let me tell you why.

A Failed First Attempt

I tried learning HTML, CSS, and javascript about a month ago. However, that didn’t go as well as I had expected, mainly due to lack of momentum. HTML was easy: it was just a way of doing markups with tags. And as many people might agree, I didn’t really think learning HTML was akin to learning a programming language. The process felt more similar to studying something like $\LaTeX$ or markdown.

Then came CSS, which turned out to be a whole different animal compared to HTML. CSS seemed simple on paper: it was just a way of stylizing HTML code so that it looks more presentable. However, the whole madness of padding, margins, classes, and ids got me rethinking about my original assessment of CSS. I ultimately ended up glossing over CSS tutorials, thinking that knowing the barebone basics—what it does and how it does it—will suffice for now.

As expected, learning javascript was more comparable to leanring Java or Python. This was mainly because javascript is, by most standards, a full-fledged programming language, featuring loops, conditional statements, variable declarations, and so on. Getting the basics was pretty easy for this reason: I was able to orient myself into programming mode and focus on learning the syntax unique to javascript. In many ways, javascript made me reminisce about Java, although there were several notable differences. This part was fun and easy.

However, when the time came to pull everything back together, I was overwhelmed. The interaction between HTML, CSS, and javascript was interesting and all, but it was all too much for me to pick up in just a few days. Tired, I decided to return back to my comfort zone of Python and Jupyter Notebooks.

Until two days ago.

I was reading about generative adversarial networks from the book Generative Deep Learning by David Foster when an idea struck me: it’s great that we can build and train a neural network, but how do we present and deploy it in actuality? Shouldn’t there be some means of delivering the model, preferably through something like an application or a website? I was able to find my answer in these questions in just a few minutes: either deploy a model by integrating it into a web application or use frameworks such as TensorFlow.js.

At this point, I realized that running into front-end work is going to be unavoidable for me. Although I’d imagine machine learning engineers or data scientists typically don’t work with web development to deploy models themselves, it is undeniable that proficiency in the front-end stack to some capacity is going to be helpful asset to have at one’s disposal. I thus garnered the courage and motivation to start my journey again with web development. This time, I had a clear sense of goal and purpose: build a web application capable of using a trained neural network or machine learning model to generate predictions and present them in a visually effective way.

Flask Web Framework

Flask is a popular Python-based web framework used for web application development. I had heard about Flask here and there from different sources, yet I never had a true idea of what it was, what it was capable of, and why it was so popular.

Python

After taking a look at some crash course videos and tutorials, I quickly understood why Flask was so popular: it was based on Python. This meant that anyone who was literate in Python could at least try Flask. This was a huge benefit since the entry barrier for Flask was considerably lower for me than it might have been had I dabbled in other web frameworks that used a different language, such as Ruby.

Of course, Flask had its own perks, and I had to get at least a basic understand of what things like GET and POST requests meant in the context of configuring a web backend. However, with some tweaking and experimentation, I realized that Flask was incredibly easy to use, at least for my purposes of realizing a simple application, mostly because it was simple, intuitive, and used Python at its core.

HTML and Bootstrap

But Python was not the only thing I needed. Because my goal was to try to build an elementary web application, I inevitably had to learn how to write and understand HTML. I had tried learning HTML a few months ago, but it was a short-lived endeavor that didn’t last for long. This time, however, I was determined to learn HTML, because it was something that I didn’t have a workaround for.

Luckily, HTML is not a difficult language to learn; it is, at its core, just a markdown language. This is a core difference between HTML and say, Python, because Python is what many would consider to be a full-fledged programming and scripting language, whereas HTML is simply a tool that can be configured to build a bare skeleton for a web page to be displayed on screen. To implement fancy features, one would need other languages and frameworks such as Javascript or React. In short, HTML is just a way of presenting information. Again, with the help of online tutorials and crash course videos, I was able to achieve some progress with my HTML literachy.

However, anyone knows that it is impossible to build a decent website with just HTML. To improve general aesthetics, I would have to learn CSS, which is a whole different animal. On top of that, building dynamic features that allows users to interact with contents on the webpage requires Javascript as mentioned earlier. I had the choice of learning them, and indeed I did pick up a few very simple Javascript syntax along the way. However, I ended up resorting to Bootstrap, a convenient yet comprehensive open source front-end framework that implements so many useful features under the hood. With the help of Bootstrap, I was able to quickly prototype a decent looking website in no time—something that would have taken weeks for me to do with vanilla Javascript and CSS.

SQL

I was also able to try out some basic SQL query statements in my first Flask mini-project. I had learned very basic SQL a few months back, yet I had never tried it out on a real project setting. Although I am still a novice in SQL, it was nice being able to see how querying could take place within a Python setting using SQL alchemy and other modules.

I used SQLite for two simple web applications: a chat logging service and a to-do application. I found SQL to be useful in building these applications because I needed to implement a way of storing information coming from users, then displaying that information back whenever necessary, which is exactly what SQL is for. For the purpose of my application, the query statements I used were incredibly simple and elementary, yet it was exciting to see that all the pieces were coming together and working well according to my expectation.

After having completed this project, I was ready to deploy my first machine learning algorithm in the form of a web application.

Deploying a Deep Learning Model

I thought deploying a deep learning model would be no different from building another simple web application, just like the ones I had built previously to familiarize myself with Flask. To an extent, this expectation held true; however, there were definitely some challenges I struggled with along the way.

Project Objective

The objective of the project was simple: assuming that there is a fully trained neural network, deploy it as a web application so that anyone could use it. For the purpose of this project, I decided to simply use an out-of-the-box model, Mobilenet, which is available on tf.keras.applications. This is somewhat of a limited approach because it does not allow for the user to retrain the model; instead, the model is set in stone, and the user can only interact with it by using it. Despite this simplified setup, however, I faced some difficulty implementing this feature mostly because of my lack of knowledge with web frameworks and the front-end in general.

Receiving an Input Image

The first challenge I faced originated from the fact that I had no knowledge of Javascript. In order to build this web application, I needed to find a way of receiving some user input—in this particular case, an image—so that the model could receive that data as input and generate a prediction. This feature itself could simply be implemented with Bootstrap HTML. However, I also wanted the name of the selected image file to be displayed on screen for the convenience of the user. To achieve this, I ended up combing through stack overflow for some time, an endeavor that was ultimately rewarded with a working code snippet that just did what I wanted. (We all know that stack overflow is the panacea to all the worries of any programmer. At this point, I almost believe that being a good programmer not only encompasses good coding skills, but also the ability to quickly perform an effective search on stack overflow to gain needed information.)

In the end, I was able to get a decent looking home page where the user could click on a button to select an image from their local storage, upload it, view the image, and ultimately pass it to the model to generate a prediction.

Displaying the Results

The next notable challenge mainly had to do with displaying the predictions generated by the model. I expected this to be a fairly straightforward process: after all, generating a prediction itself could simply be achieved with the predict() function available in Keras. I expected using matplotlib to display this result to be simple, but unfortunately, HTML webpages are not Jupyter notebooks; I could not simply use plt.show() to display a plot on the screen.

In the end, after some trial and error and performing countless searches on stack overflow, I managed to get a working function that achieved my goal:

def create_plot(label, estimate):
    output = BytesIO()
    fig = Figure(figsize=(16,10), dpi=300, tight_layout=True)
    axis = fig.add_subplot(1, 1, 1)
    axis.bar(label, estimate, color='#007bff')
    FigureCanvas(fig).print_png(output)
    image_string = "data:image/png;base64,"
    image_string += base64.b64encode(output.getvalue()).decode('utf8')
    return image_string

To this day, I do not quite understand all of what is going on in this code snippet. However, I do know that there is some encoding and decoding taking place so that the created image could be delivered to Flask’s render_template function as a parameter.

I also created some fail-safes to ensure that the image that was provided by the user was of supported format. This could simply be achieved with the following function:

ALLOWED_EXTENSIONS = {'png', 'jpg', 'jpeg'}

def check_extension(file_name):
    return '.' in file_name and \
           file_name.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS

While this is not necessary, having this function helped me deal with problems that might occur if the user were to input some odd unsupported image type, such as a .pdf file.

Finally after much tweaking and coding, this is what I had when I uploaded an image of a cute dog to the web application!

The result:

You know how happy I was when all of this was done after a week of intensive Flask-ing, HTML-ing, and Boostrap-ing.

Conclusion

This was one of the first projects I had worked on independently aside from the small Jupyter notebook-based programming I did to write some of the posts on this blog. Honestly, it was a fantastic and much-needed experience because I had not done anything this project-based for a long time. In fact, the last time I had worked on a project was in high school for my AP Computer Science final.

Nostalgic reflections aside, I feel like I was able to add a valuable kit to my available technology stack, and that is Flask. Flask is incredibly easy to pick up, especially if you come from a Python background. Setting up a Flask application is also mind-blowingly simple, and this was one of the reasons I decided to use Flask instead of other frameworks, say Django. Don’t get me wrong: I have heard so much great things about Django that I’m considering learning it on top of Flask, but for a quick, small project as this one, Flask was the perfect fit.

Now that I know how to deploy models, albeit in an elementary fashion, I guess I can continue with my quest on deep learning with the reassurance that I (kind of) know how to deliver ML models to individual users through an established pipeline.