We study generation of a 3D model of sport events on a large field with visual hull using multi-view videos that were taken with several cameras sparsely arranged surrounding the field. To efficiently generate the model, we introduce a hierarchical octree structure to construct the model and we also propose a method to delete inside of the model. Moreover, for efficiently handling occlusions, we propose a robust curve fitting method using Fourier series to correctly capture the color changes of each voxel along the direction from which it is seen. We can put a correct color to each voxel for any given viewpoint with this method.