AI Free Code

Convert hand-drawn design to HTML

In this tutorial, we’ll be building a program that is used to convert the hand-drawn design into HTML, by using an algorithm called sketch2code. The converted HTML will be adjusted based on the device-width, and responsive across devices. So, to boost the conversion, the AI algorithm allows the use of the imported designs for continuous learning. 

Process Flow

Transformation method of a handwritten image to code, Implementation of this approach is shown below:

  • 1. The user first uploads an image via the website.

  • 2. A model named custom vision determines what HTML components appear in the image and where they are positioned.

  • 3. In the predicted elements, a handwritten text generation service reads the text inside.

  • 4. A model algorithm uses spatial knowledge from all of the projected product bounding boxes to create a grid structure that matches all.

  • 5. An engine of generation HTML uses all these pieces of knowledge to create an HTML markup code that represents the outcome of an image.

The architecture involved to convert the hand-drawn design to HTML

  • 1. A Microsoft Custom Vision Model : This model was trained with images of various handwriting styles tagging details on the most popular HTML elements such as Images, TextBox, and Buttons.

  • 2. A Microsoft computer vision service : A service based on computer vision is used for defining the text written into a design feature.

  • 3. Azure Blob storage : All steps involved in the process of creating HTML are processed, including original image, effects of predictions, and material grouping of the model.

  • 4. An Azure function: It serves as the backend entry point that manages the cycle of generation by communicating with all resources.

  • 5. An Azure web app: User front-end to encourage you to upload a new template and see the effects of the created Code. Those elements form the following architecture:

Fig: Architecture of Sketch2code solution

Note: You can create your Custom Vision Project at


Convert hand-drawn design to HTML


2 thoughts on “Convert hand-drawn design to HTML

Leave a Reply

Your email address will not be published. Required fields are marked *