colmap处理流程

Gaussian Splatting中COLMAP处理流程

• feature_extractor
• exhaustive_matcher
• mapper
• image_undistorter

feature_extractor

命令：

colmap feature_extractor \
        --database_path /home/yugrp01/dataset//distorted/database.db \
        --image_path /home/yugrp01/dataset//input \
        --ImageReader.single_camera 1 \
        --ImageReader.camera_model OPENCV \
        --SiftExtraction.use_gpu 1

作用：提取图像特征，保存在database.db中，默认提取SIFT特征

exhaustive_matcher

命令：

colmap exhaustive_matcher \
        --database_path /home/yugrp01/dataset//distorted/database.db \
        --SiftMatching.use_gpu 1

作用：建立SIFT匹配，仍保存在database.db中

mapper

命令：

colmap mapper \
        --database_path /home/yugrp01/dataset//distorted/database.db \
        --image_path /home/yugrp01/dataset//input \
        --output_path /home/yugrp01/dataset//distorted/sparse \
        --Mapper.ba_global_function_tolerance=0.000001

作用：相机位姿求解与优化，在sparse文件夹下生成多个文件夹，一个文件夹对应一个相机，可以使用命令

colmap model_converter \
        --input_path /home/yugrp01/dataset//distorted/sparse/0 \
        --output_path /home/yugrp01/dataset//distorted/sparse \
        --output_type TXT

将sparse/0中的.bin文件转换为.txt文件
mapper生成的文件为：

• cameras.bin

  # Camera list with one line of data per camera:
  #   CAMERA_ID, MODEL, WIDTH, HEIGHT, PARAMS[]
  # Number of cameras: 1
  1 OPENCV 720 1280 1120.1152718874055 1173.5453696531381 360 640 0.056079336784625386 -0.085614936918791046 -0.0035479370208935067 -0.00036881030187927946

  内容为：相机ID，相机模型(OPENCV)，图像宽度，图像高度，相机内参
• images.bin

  # Image list with two lines of data per image:
  #   IMAGE_ID, QW, QX, QY, QZ, TX, TY, TZ, CAMERA_ID, NAME
  #   POINTS2D[] as (X, Y, POINT3D_ID)
  # Number of images: 151, mean observations per image: 899.41059602649011
  139 0.97558110892036964 0.024064873414301063 0.19083814762661425 0.10603387758494046 2.0978674295044142 1.9014249718018907 -4.6688458262732428 1 0141.jpg
  342.20053100585938 2.5576851367950439 -1 294.03756713867188 8.3915014266967773 -1 710.8546142578125 9.4258575439453125 -1 320.52203369140625 22.885553359985352 -1 320.52203369140625 22.885553359985352 -1 333.97128295898438 25.935688018798828 -1 333.97128295898438 25.935688018798828 -1 707.79693603515625 26.888103485107422 -1 406.88790893554688 29.064554214477539 -1 320.9920654296875 30.22673225402832 -1 320.9920654296875 30.22673225402832 -1 294.0234375 31.514423370361328 -1 324.99057006835938 38.868644714355469 -1 324.99057006835938 38.868644714355469 -1 331.5408935546875 62.630966186523438 -1 331.5408935546875 62.630966186523438 -1 292.57781982421875 69.072616577148438 -1 292.57781982421875 69.072616577148438 -1 302.60647583007812 71.223625183105469 -1 302.60647583007812 71.223625183105469 -1 317.67379760742188 74.406044006347656 -1 317.67379760742188 74.406044006347656 -1 286.7628173828125 74.998977661132812 -1 286.7628173828125 74.998977661132812 -1 329.06219482421875 76.793952941894531 -1 329.06219482421875 76.793952941894531 -1 312.201416015625 80.79327392578125 -1 312.201416015625 80.79327392578125 -1 309.02401733398438 ...

  内容为：图像ID，相机位姿，相机ID，图像名称，图像特征点(包括3D点ID，大多数是-1)
• points3D.bin

  # 3D point list with one line of data per point:
  #   POINT3D_ID, X, Y, Z, R, G, B, ERROR, TRACK[] as (IMAGE_ID, POINT2D_IDX)
  # Number of points: 22731, mean track length: 5.9747041485196428
  22664 9.3397715374614645 -0.5320455884139641 -0.35230666689830648 174 173 169 0.54436590619680714 11 963 3 47

  内容为：3D点ID，3D点坐标，3D点颜色，3D点误差，3D点在图像中的投影(图像ID，特征点ID)

image_undistorter

命令：

colmap image_undistorter \
        --image_path /home/yugrp01/dataset//input \
        --input_path /home/yugrp01/dataset//distorted/sparse/0 \
        --output_path /home/yugrp01/dataset/ \
        --output_type COLMAP

作用：去畸变，生成去畸变后的图像，保存在/home/yugrp01/dataset/中

替代COLMAP

COLMAP在GS中的应用仅是通过特征点匹配来获得相机位姿，可以使用其他方法代替COLMAP，如ORB-SLAM2，VINS-Mono等
具体代码位于scene/dataset_readers.py的readColmapSceneInfo函数：

try:
    cameras_extrinsic_file = os.path.join(path, "sparse/0", "images.bin")
    cameras_intrinsic_file = os.path.join(path, "sparse/0", "cameras.bin")
    cam_extrinsics = read_extrinsics_binary(cameras_extrinsic_file)
    cam_intrinsics = read_intrinsics_binary(cameras_intrinsic_file)
except:
    cameras_extrinsic_file = os.path.join(path, "sparse/0", "images.txt")
    cameras_intrinsic_file = os.path.join(path, "sparse/0", "cameras.txt")
    cam_extrinsics = read_extrinsics_text(cameras_extrinsic_file)
    cam_intrinsics = read_intrinsics_text(cameras_intrinsic_file)

reading_dir = "images" if images == None else images
cam_infos_unsorted = readColmapCameras(cam_extrinsics=cam_extrinsics, cam_intrinsics=cam_intrinsics, images_folder=os.path.join(path, reading_dir))
cam_infos = sorted(cam_infos_unsorted.copy(), key = lambda x : x.image_name)

cam_infos中是每张图的位姿和图片参数，比如cam_infos[:3]为：

[CameraInfo(uid=1, R=array([[ 0.38001315, -0.23005276,  0.89591614],
[ 0.23318819,  0.96112063,  0.14788647],
[-0.89510518,  0.15271826,  0.41888406]]), T=array([-4.40688372,  0.55051116, -2.53033356]), FovY=0.9846972407709619, FovX=0.6130229182895368, image=<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=709x1259 at 0x7F54D3CCA110>, image_path='/home/yugrp01/dataset/images/0001.jpg', image_name='0001', width=709, height=1259), 
CameraInfo(uid=1, R=array([[ 0.39393992, -0.22690239,  0.89068886],
[ 0.23024742,  0.96251379,  0.14336435],
[-0.88983002,  0.14860187,  0.43141629]]), T=array([-4.41738619,  0.52536371, -2.47282881]), FovY=0.9846972407709619, FovX=0.6130229182895368, image=<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=709x1259 at 0x7F54D3CCA050>, image_path='/home/yugrp01/dataset/images/0002.jpg', image_name='0002', width=709, height=1259), C
ameraInfo(uid=1, R=array([[ 0.4211666 , -0.22125352,  0.87958261],
[ 0.22593051,  0.96481198,  0.13451114],
[-0.87839291,  0.14207294,  0.45633451]]), T=array([-4.45405079,  0.48200487, -2.35331303]), FovY=0.9846972407709619, FovX=0.6130229182895368, image=<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=709x1259 at 0x7F54D3CB3F50>, image_path='/home/yugrp01/dataset/images/0003.jpg', image_name='0003', width=709, height=1259)]

其中，CameraInfo包含了相机位姿，图片，图片路径等信息，uid为相机ID，R为旋转矩阵，T为平移向量，FovY和FovX为视场角，image为图片，image_path为图片路径，image_name为图片名称，width和height为图片宽高，仅需自行输入R，T即可