The puzzle assembly problem has many application areas such as restoration and reconstruction of archeological findings, repairing of broken objects, solving jigsaw type puzzles, molecular docking problem, etc.. This paper presents a new approach to the puzzle assembly problem that is based on using textural features and geometrical constraints. The texture of a band outside the border of pieces is predicted by inpainting and texture synthesis methods. The confidence of this process is also defined. Feature values are derived from these original and predicted images of pieces. A combination of the feature and confidence values is used to generate an affinity measure of corresponding pieces. The optimization of total affinity gives the best assembly of the puzzle. Experimental results are presented as real and artificial data. An application, V-Stitch is also developed, with the purpose of providing a real-time virtual reality software system with an intuitive interface for using semi-automatic algorithms developed for the arrangement of archeological sherds