| 研究生: |
賴雅鈴 Lai, Ya-Ling |
|---|---|
| 論文名稱: |
以感知損失神經網路平滑化樂高平板磚之影像樂高風格化技術 2D Lego Flat Tiles Generation with Perceptual Loss Neural Network |
| 指導教授: |
紀明德
Chi, Ming-Te |
| 口試委員: |
葉奕成
Yeh, I-Cheng 王科植 Wang, Ko-Chih |
| 學位類別: |
碩士
Master |
| 系所名稱: |
資訊學院 - 資訊科學系 Department of Computer Science |
| 論文出版年: | 2023 |
| 畢業學年度: | 111 |
| 語文別: | 中文 |
| 論文頁數: | 51 |
| 中文關鍵詞: | 樂高 、編碼器-解碼器網路架構 、拼貼 |
| 外文關鍵詞: | LEGO, Encoder-decoder network architecture, Collage |
| 相關次數: | 點閱:18 下載:0 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
樂高公司持續推出新的系列及不同種類的磚,這樣的多樣性也使得樂高深受大人小孩的喜愛,對於一些新推出的樂高系列,相對較少有研究進行探討,但也會有與其他領域,像是拼貼、排列、像素化問題等或與樂高研究議題相關的地方。像素化藝術方面的研究一直以來都非常受歡迎,但這樣的風格就是每個區域都為方形,對於一些圖形較圓滑的地方無法很好地表現出來。本研究的樂高豆豆系列則是將畫素以實體元件表現出來,而除了方形的磚,也有一些帶有較圓滑邊緣的磚。但當我們要以人工的方式去拼一個形狀時,在磚形狀及顏色的選擇上,就會花費非常多的時間,如果要拼的東西越大,所花費的時間也就更久,會浪費許多勞力和時間。
為了解決這些問題,本研究首先嘗試將編碼器-解碼器架構的網路與二維樂高平板磚建構相結合。以樂高平板磚中的方形磚及帶圓滑邊緣的磚轉換為圖片作為輸入,並透過給定損失函數,將現有的照片馬賽克神經網路研究延伸至樂高平板磚的組合問題。同時,我們也針對輸入圖形及生成的圖片,做一系列的比較及分析,證明此系統的有效性。
LEGO continues to release new series and different types of bricks, which has made it popular with both adults and children. However, there has been relatively little research on some of the newer Lego series, but there are also related studies in other areas such as puzzles, arrangements, pixelation, and other Lego research topics. Pixel art research has always been very popular, but this style is characterized by square regions, which makes it difficult to represent smoother shapes. The Lego Dots series studied in this research is just like expressing pixels as physical components, and in addition to square bricks, there are also bricks with smoother edges. However, when we try to manually assemble a shape with bricks, it takes a lot of time to choose the shape and color of the bricks, and the larger the thing we want to assemble, the longer it takes, wasting a lot of labor and time.
In order to solve these problems, this research first attempts to combine the network of encoder-decoder architecture with the construction of two-dimensional Lego flat bricks. Taking square bricks and bricks with rounded edges in Lego flat bricks as input, and through a given loss function, the existing photomosaic neural network research is extended from the collage problem to the combination problem of Lego flat bricks. At the same time, we also made a series of comparisons and analyzes on the input images and the generated images to show the effectiveness of this system.
第一章 緒論 1
1.1 研究動機 1
1.2 問題描述 3
1.3 論文貢獻 3
1.4 論文章節架構 4
第二章 相關研究 5
2.1 圖片馬賽克問題 5
2.2 拼貼問題 6
2.3 像素化藝術 8
第三章 研究方法 10
3.1 系統流程 10
3.2 圖片資料集 11
3.3 資料預處理 12
3.4 編碼器與解碼器架構 14
3.5 選擇圖塊 15
3.6 感知損失 15
3.7 顏色處理 17
3.8 量化分析 18
第四章 結果與限制 25
4.1 研究細節 25
4.2 實驗數據比較 25
4.2.1 與Baseline方法比較 25
4.2.2 輸入圖塊集差異 26
4.2.3 組裝結果大小 28
4.2.4 選用尺度不同 29
4.2.5 損失函數使用不同 31
4.3 案例分析 32
4.4 結果 33
4.4 環境 40
4.5 限制 40
第五章 結論與未來展望 42
5.1 結論 42
5.2 未來展望 43
參考文獻 44
附錄 46
[ 1 ] Di Blasi, G., Gallo, G., & Petralia, M. (2005, September). Puzzle image mosaic. In Proc.
IASTED/VIIP (pp. 33-37).
[ 2 ] Zou, C., Cao, J., Ranaweera, W., Alhashim, I., Tan, P., Sheffer, A., & Zhang, H. (2016). Legible compact calligrams. ACM Transactions on Graphics (TOG), 35(4), 1-12.
[ 3 ] Kwan, K. C., Sinn, L. T., Han, C., Wong, T. T., & Fu, C. W. (2016). Pyramid of arclength descriptor for generating collage of shapes. ACM Trans. Graph., 35(6), 229-1.
[ 4 ] Chen, M., Xu, F., & Lu, L. (2019). Manufacturable pattern collage along a boundary. Computational Visual Media, 5, 293-302.
[ 5 ] Akiyama, O. (2017). ASCII art synthesis with convolutional networks. In Proc. NIPS Workshop Mach. Learn. Creativity Design (pp. 1-7).
[ 6 ] Tesfaldet, M., Saftarli, N., Brubaker, M. A., & Derpanis, K. G. (2018). Convolutional Photomosaic Generation via Multi-Scale Perceptual Losses. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops (pp. 0-0).
[ 7 ] Kim, J., & Pellacini, F. (2002). Jigsaw image mosaics. ACM Transactions on Graphics, 21(3), 657-664.
[ 8 ] Xu, P., Ding, J., Zhang, H., & Huang, H. (2019). Discernible image mosaic with edge-aware adaptive tiles. Computational Visual Media, 5, 45-58.
[ 9 ] Gerstner, T., DeCarlo, D., Alexa, M., Finkelstein, A., Gingold, Y., & Nealen, A. (2012, June). Pixelated image abstraction. In Proceedings of the Symposium on Non-Photorealistic Animation and Rendering (pp. 29-36).
[ 10 ] Inglis, T., & Kaplan, C. S. (2012). Pixelating vector line art. SIGGRAPH Posters, 108.
[ 11 ] Shang, Y., & Wong, H. C. (2021). Automatic portrait image pixelization. Computers & Graphics, 95, 47-59.
[ 12 ] Huang, M. R., & Lee, R. R. (2015). Pixel Art Color Palette Synthesis. In Information Science and Applications (pp. 327-334). Springer Berlin Heidelberg.
[ 13 ] Orchard, J., & Kaplan, C. S. (2008, June). Cut-out image mosaics. In Proceedings of the 6th international symposium on Non-photorealistic animation and rendering (pp. 79-87).
[ 14 ] Shen, I. C., & Chen, B. Y. (2021). Clipgen: A deep generative model for clipart vectorization and synthesis. IEEE Transactions on Visualization and Computer Graphics, 28(12), 4211-4224.
[ 15 ] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
[ 16 ] Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612.
[ 17 ] Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 586-595).
[ 18 ] Johnson, J., Alahi, A., & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14 (pp. 694-711). Springer International Publishing.
[ 19 ] Sacht, L. (2022). Structure-aware bottle cap art. Computers & Graphics, 107, 277-288.
[ 20 ] Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Süsstrunk, S. (2010). Slic superpixels (No. REP_WORK).
[ 21 ] Han, C., Wen, Q., He, S., Zhu, Q., Tan, Y., Han, G., & Wong, T. T. (2018). Deep unsupervised pixelization. ACM Transactions on Graphics (TOG), 37(6), 1-11.
[ 22 ] Doyle, L., Anderson, F., Choy, E., & Mould, D. (2019). Automated pebble mosaic stylization of images. Computational Visual Media, 5, 33-44.
[ 23 ] Hsiang-Yu Wang, Ming-Te Chi, Mapping 2D Lego Construction into Tiling Problem with Graph Neural Network (2022).