Generating Website Codes from Images

My Ha Lai Nguyen, Thien Ngoc Huynh


Brainstorm website layout ideas usually start with everyone
giving out their mockups, and all the team members will discuss
to finalize the layout of the user interface. Once a vision of that
mockup is accepted, it is given to the designer to sketch it digitally
on computer software (i.e., Photoshop, Figma, Sketch). When
the designer completes, the developer based on the final design
to code the UI/UX of the website. As we can see, the process
requires three stages, which can be time-consuming. Therefore,
if anyone has an idea for the professional website layout, they can
visualize it by drawing on sketches. However, it can be impossible
for them to make a usable website without designers and website
developers. Due to that reason, our primary goal in this paper
is to help individuals transform their hand-drawn sketch images
into a website that can be deployed. To achieve that goal, we
present two approaches: classical computer vision techniques
and the other using a deep learning model to detect the sketch
and execute the conversion. Furthermore, our evaluation shows
that deep learning is the most promising direction. Still, classical
techniques also improve the model’s input data by applying it in
the pre-processing image.
Index Terms—Keywords: Computer vision, Hand-drawn
sketch, UI/UX, Deep learning


J. A. Landay and B. A. Myers, Sketching interfaces: toward more human interface design. Computer 34.3, pages 56–64, March 2001.

James Lin et al., DENIM: Finding a Tighter Fit Between Tools and Practice for Web Site Design. Computer 34.3, pages 510–517, April 2000.

T. A. Nguyen and C. Csallner, Reverse Engineering Mobile Application User Interfaces with REMAUI (T). 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 248–259, November 2015.

R. Ingle, Yasuhisa Fujii, Thomas Deselaers, Jonathan Baccash, and Ashok Popat, A scalable handwritten text recognition system. pages 17–24, September 2019.

Alex Robinson, Sketch2code: Generating a website from a paper mockup May 2019.

Liao, Tianli, and Nan Li, “Single-Perspective Warps in Natural Image Stitching.”, 7 Mar. 2018,

Beltramelli and Tony, pix2code:Generating code from a graphical user interface screenshot. arXiv preprint arXiv:1705.07962, 2017.

Durjoy Sen Maitra, Ujjwal Bhattacharya, and Swapan Parui, Cnn based common approach to handwritten character recognition of multiple scripts. pages 1021–1025, August 2015.

T. Y. Zhang and C. Y. Suen, A fast parallel algorithm for thinning digital patterns. Commun. ACM, 27:236–239, March 1984.

Muneeb Ahmed and Jeff Wheeler, Generation of slides from hand-drawn sketches. 2014.

Yuntian Deng, Anssi Kanervisto, and Alexander Rush, What you get is what you see: A visual markup decompiler. September 2016.

L. Donati, S. Cesano, and A. Prati, A complete hand-drawn sketch vectorization framework. Multimedia Tools and Applications, pages 1–31 February 2019.

Graham Finlayson, Steven Hordley, and Mark Drew, A fast parallel algorithm for thinning digital patterns. Commun. ACM, 2353:129, 2002.

Manuel J. Fonseca and Joaquim A. Jorge, Using fuzzy logic to recognize geometric shapes interactively. IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 2:1–20, May 2000.

Manuel J. Fonseca and Joaquim A. Jorge, Multi-domain recognition of hand-drawn diagrams using hierarchical parsing. Multimodal Technologies and Interaction, August 2020.

L. Zhang Y. Rui J. Wu, C. Wang, AAAI’14: Proceedings of the TwentyEighth AAAI Conference on Artificial Intelligence, page 951–957, July 2014.

Joaquim A. Jorge and Manuel J. Fonseca., simple approach to recognise geometric shapes interactively. Graphics Recognition, Recent Advances, Third International Workshop, GREC’99 Jaipur, India, September 26-27, 1999, Selected Papers, 1941:266–276, January 1999.

Felipe Such, Dheeraj Peri, Frank Brockler, Paul Hutkowski, and Raymond Ptucha, Fully convolutional networks for handwriting recognition. July 2019.

Zhenbang Sun, Changhu Wang, Liqing Zhang, and Lei Zhang,

Sketch2tag: Automatic hand-drawn sketch recognition. MM 2012 - Proceedings of the 20th ACM International Conference on Multimedia, pages 1255–1256, October 2012.


Copyright (c) 2022 REV Journal on Electronics and Communications

Copyright © 2011-2022
Radio and Electronics Association of Vietnam
All rights reserved