Automating free photogrammetry: Scripts I use

[updated 17/12/17 to get rid of unnecessary meshlab portion]

[updated 01/04/18 – Updated scripts for version 3.4+ of COLMAP]

In a vain attempt to be useful, I want to share the script I’m currently using to automate my photogrammetry workflow.

I’m using COLMAP in conjunction with openMVS on Windows 10 at the moment, and I feel that gives me robust results, in a good amount of time.  You do need a decent computer with an Nvidia graphics card for this to work.

Essentially, I’ve automated my workflow so I don’t have to open up the COLMAP GUI each time, and then type in openMVS commands at the command prompt.  I’ve written the following script so that I can just take a folder full of photos, drop my batch file in there, double click it and come back a little later to a fully textured model (assuming it works without hiccup).

So, if you’d like to take advantage of this, the first thing you need to do is download both COLMAP (link) and the openMVS windows binaries (link).

Let’s unzip the colmap and openMVS folders into our downloads folder, and install meshlab.

Create an empty text file with notepad, and paste in the following:

::These parameters are specific to computer 

::Store current Directory: 
set currDir=%CD% 

:: Set colmap directory:
set colDir=C:\Users\Peter\Downloads\COLMAP-dev-windows\bin

:: Set openMVS directory
set oMVS=C:\Users\Peter\Downloads\openMVS_sample-0.7a

:: Set Working Directory (windows)
set workDir=D:\Output

mkdir %workDir% 
copy *.jpg %workDir%\ 
cd /d %workDir%

%colDir%\feature_extractor --database_path database.db --image_path .
%colDir%\exhaustive_matcher --database_path database.db
mkdir sparse
%colDir%\mapper --database_path %workDir%\database.db --image_path . --export_path %workDir%\sparse
%colDir%\model_converter.exe --input_path sparse\0 --output_path model.nvm --output_type NVM
%oMVS%\InterfaceVisualSFM.exe model.nvm
%oMVS%\DensifyPointCloud.exe model.mvs
%oMVS%\ReconstructMesh.exe model_dense.mvs
%oMVS%\RefineMesh.exe --resolution-level 1 model_dense_mesh.mvs
%oMVS%\TextureMesh.exe --export-type obj model_dense_mesh_refine.mvs
copy *.obj %currDir%\model\
copy *.mtl %currDir%\model\
copy *.png %currDir%\model\

::Uncomment these lines in to delete working directory
::cd /d %currDir%
::RD /S /Q %workDir%

Alter the script accordingly (see below) then change the extension of the file from *.txt to *.bat.

What this script does, is copy all the images in the same directory as the script into a working directory, run the COLMAP commands there, then the openMVS commands, then uses meshlab to convert the ply to obj.  Let me talk you through it, because there’s some bits you’ll need to change according to your setup.

Variables

::These parameters are specific to computer

::Store current Directory:
set currDir=%CD%

:: Set colmap directory:
set colDir=C:\Users\Peter\Downloads\COLMAP-dev-windows\bin

:: Set openMVS directory
set oMVS=C:\Users\Peter\Downloads\openMVS_sample-0.7a

:: Set Working Directory (windows)
set workDir=D:\Output

::Meshlab Directory:
set meshDir=”C:\Program Files\VCG\MeshLab”

mkdir %workDir%
copy *.jpg %workDir%\
cd /d %workDir%

This sets the directories for the colmap and openmvs binaries so that I didn’t have to type them out multiple times in the script, and so it’s easier to change depending on which computer it’s running on.  If you’ve left the two folders in your Downloads folder, you can probably just change the username from ‘Peter’ to whatever your username is.

I also put in the default install directory for Meshlab, as we’ll be using the meshlabserver.exe in there later on.

I’ve set my working directory (%workDir%) to D:\Output.  The script will run everything from there so that a) I always know where intermediate files will be saved, b) by copying the current images, it reduces the chance of accidentally deleting something vital, and c) I keep my image sets in onedrive; by moving the working directory outside onedrive, I make sure that it’s not trying to upload all the intermediate steps.

Run COLMAP

%colDir%\feature_extractor –database_path database.db –image_path .
%colDir%\exhaustive_matcher –database_path database.db
mkdir sparse
%colDir%\mapper –database_path %workDir%\database.db –image_path . –export_path %workDir%\sparse
%colDir%\model_converter.exe –input_path sparse\0 –output_path model.nvm –output_type NVM

The next few lines work through the colmap executables in order to: Extract features, match features, create depth mapps, and save the matched cameras as an NVM file (VisualSFM file format, needed by openMVS below).  The instructions for this were taken from the COLMAP executables documentation.

Run openMVS

%oMVS%\InterfaceVisualSFM.exe model.nvm
%oMVS%\DensifyPointCloud.exe model.mvs
%oMVS%\ReconstructMesh.exe model_dense.mvs
%oMVS%\RefineMesh.exe model_dense_mesh.mvs
%oMVS%\TextureMesh.exe model_dense_mesh_refine.mvs

This runs, in turn, various commands to convert the nvm file to openMVS, create a dense point cloud, then reconstruct, refine, and texture the mesh.  On lower memory systems, it tends to be the refine mesh stage that really brings a computer to it’s knees, and if that’s happening you might want to look into the openMVS documentation to see how to scale down images for this step.

 

Clean up

The last few lines are optional:

::Uncomment these lines in to delete working directory
::cd /d %currDir%
::RD /S /Q %workDir%

But these will delete the working directory automatically.  It’s actually better to do this manually, because if something fails half way through, these commands will delete all those intermediate files and you won’t be able to see where it failed.

Have fun!

I hope this is useful to people that have a lot of photogrammetry to do, and don’t want to manually run every stage for every dataset.  I offer this of course with absolutely zero support!  If it doesn’t work or makes your computer blow up, not my fault!

Advertisements

35 thoughts on “Automating free photogrammetry: Scripts I use

  1. Nicely explained, I wrote a script to automatise openMSV workflow as well but ended up having to control each step on the way. My laptop is quite limited and I have to crop the mesh before refine otherwise it takes ages!
    But I really understand and approve the process!

    You might try to export directly the obj from openMSV without going through meshlab by adding the option “–export-type obj” at the Texturemesh.
    I don’t know if there is a difference with meshlab but it works fine for me.

    Next step: automating picture acquisition!

    1. Ah, you’re absolutely right – I found openMVS could output straight to obj a couple of weeks back and edited my own script accordingly. I’ll put up a new post soon summarizing everything and including it.

      Great idea cropping before refining – I’ve a couple of models even my workstation falls over with.

      Yes… I’ve been looking at automated turntables, but can’t seem to find the time (or a good connection to a sony nex-6) to set it up. If you find a good guide, let us know!

      1. Indeed, I also wanted to make one but in my case I don’t even have a turntable to start with. However I have all the electronic to control it! There are lot of guide but often involved electronic design. It’s basically a motor and arduino but many people are not comfortable with it. Also a motorized turntable with USB seems to cost +100 dollars which is amazingly expensive for what it is.

        It would be nice to have a solution which is more “plug and play” if someone is not familiar with electronic device.
        If your shutter speed is fast, maybe can just buy a cheap 360 turntable and doing a sort of timelapse with the camera?

        To control your sony, did you try digicamcontrol? I see it is supported.
        With it you can also add a script and play with arduino, so might be possible to start the turntable and once the picture taken, run your .bat file.

      2. Hi pfalkingham

        I ran this code on my power shell window i have change the user name but its showing error following is the error.

        %colDir%\exhaustive_matcher : The module ‘%colDir%’ could not be loaded. For more information, run ‘Import-Module %colDir%’.
        At line:10 char:1
        + %colDir%\exhaustive_matcher –database_path database.db
        + ~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo : ObjectNotFound: (%colDir%\exhaustive_matcher:String) [], CommandNotFoundException
        + FullyQualifiedErrorId : CouldNotAutoLoadModule

        %oMVS%\InterfaceVisualSFM.exe : The module ‘%oMVS%’ could not be loaded. For more information, run ‘Import-Module %oMVS%’.
        At line:14 char:1
        + %oMVS%\InterfaceVisualSFM.exe model.nvm
        + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo : ObjectNotFound: (%oMVS%\InterfaceVisualSFM.exe:String) [], CommandNotFoundException
        + FullyQualifiedErrorId : CouldNotAutoLoadModule

        %oMVS%\DensifyPointCloud.exe : The module ‘%oMVS%’ could not be loaded. For more information, run ‘Import-Module %oMVS%’.
        At line:15 char:1
        + %oMVS%\DensifyPointCloud.exe model.mvs
        + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo : ObjectNotFound: (%oMVS%\DensifyPointCloud.exe:String) [], CommandNotFoundException
        + FullyQualifiedErrorId : CouldNotAutoLoadModule

        can you please suggest me something so that i can get out this.

      3. Did you put it in a batch file? Running it directly in powershell might spit out this error. Make sure you declare %colDir%, and put the whole lot in a batch file.

    2. Hi Mesheritage, I read your comment on cropping the mesh before refinement. That’s exactly what I need, but unfortunately I can´t figure out how to do it. First I thought I have to edit the .ply files that are written after each step, but this doesn´t make a difference. So maybe you could give me a hint how to do this 🙂 Many thanks.

      1. You have to pass the original MVS file plus edited ply file. I can’t find the instructions right now (can’t remember where I saw them) – I’ll see if I can find them then edit this comment if I do.

  2. Just stumbled on your blog a few days ago and I’m really enjoying reading your articles! I did play around with 3D scanning about a year ago, but never go a satisfying workflow.
    Took me some while to set it up, but I’ve got the colmap and openMVS conjunction up and running today.
    No matter what I did I could not get the precompiled version of openMVS to work on my Windows 10 machine, so I had to build it from scratch. And because I’ve got access to a strong pc just for rendering and computing, which is running a linux distribution I decided to actually setup and build colmap and openMVS on that machine.
    The building and installing worked nearly flawless, which is way better than I can say about my experience of getting it running on windows..

    I then rewrote your .bat file as a .sh file to run on linux and after about a day it now working. And I stunned by the speed and high quality of results.

    If anyone wants to have my .sh file just ask. It was not too hard to rewrite it as bash, as .bat and .sh syntax are pretty similar, but anyways if theres interest I would share it. 😀

    Only Problem I encountered: The RefineMesh step seemed to crash on my gpu, so I had to add the “–use-cuda 0” argument to run it on the cpu. It’s slower, but it’s working just fine otherwise.

    So finally, Thanks a lot for this and your other articles! Really helpful! Keep the good stuff coming 🙂

      1. Hey, sure. It’s only the .sh file, so linux only, I think. I have not touched the .bat. But I guess you could improve it by changing the line “%oMVS%\RefineMesh.exe model_dense_mesh.mvs” to “%oMVS%\RefineMesh.exe model_dense_mesh.mvs –use-cuda 0” if it crashes at the refine step.

        I just uploaded my .sh file to Pastebin. Here you go: https://pastebin.com/h2dfUUYY

        Cheers,
        Kai

  3. http://www.creativeshrimp.com/free-photo-scanning-tutorial.html

    first is about to insytall and run all soft in linux 🙂 VSFM, Colmap, openMVS…
    if nVidia+Intel are used on computer/laptop the CUDA is activated by installing linux drivers for Intel and nVidia:
    Intel:
    https://01.org/linuxgraphics/downloads/intel-graphics-update-tool-linux-os-v2.0.5
    optional:
    https://launchpad.net/~oibaf/+archive/ubuntu/graphics-drivers
    nVidia:
    https://launchpad.net/~graphics-drivers/+archive/ubuntu/ppa

    + eventually install CUDA Toolkit if extra needed, from Ubuntu repository

    then by running nVidia x-server settings in Ubuntu there is a tab option ‘nVidia Prime’ where Intel or nVidia can be selected and then logout/login to system = done 🙂 in my opinion linux environment may be faster than Windows but it’s just a hypothesis.

    nice blog and topic btw, hope it’ll be continued if new stuff arrives. I’m just about to check the photogrammetry now 🙂

  4. Little help, trying to execute script, does notrecognize “feature_extractor”. Powershell says it is not a recognizable internal or external command. Please help I am new, and seek help..

    1. Yeah, that’s because colmap v3.4 changed things from multiple executables to arguments to a single executable, so for instalce instead of :

      %colDir%\feature_extractor –database_path database.db –image_path .

      You want to put

      %colDir%\colmap.exe feature_extractor –database_path database.db –image_path .

  5. There is a small error in bat file you showing here.

    %colDir%\colmap feature_extractor –database_path database.db –image_path .
    %colDir%\colmap exhaustive_matcher –database_path database.db
    mkdir sparse
    %colDir%\colmap mapper –database_path %workDir%\database.db –image_path . –export_path %workDir%\sparse
    %colDir%\colmap model_converter.exe –input_path sparse\0 –output_path model.nvm –output_type NVM

    You forgot to add “colmap” in each line.

  6. With Colmap 3.4 on Windows, as well as adding colmap.exe to the command lines, the colmap dll files need to be in the current path. I did this by copying the content of the colmap/lib directory to the colmap/bin directory.

    The .png files are at undistorted_images\*.png

    Thanks for a very useful script

    Alan

  7. Ohhhh so it currently only works in colmap versions 3.3 and below, and the dlls from lib need to be copied to bin. That explains a lot. Had a frustrating old time before I read these comments.

  8. Nice and fast. I tried colmap alone, but the files wasn’t useable. The script itself need some updates to get it to work, but most things are already in the comments here.
    I have updated the script, so that it works for me, it can be found on my website under 3D-Gedöns -> 3D Scanning, but only witch german description. If you, Dr Peter Falkingham, do not want that, give me a short feedback. If someone will give it a try, please don’t forget to copy the files in from the lib folder to the bin folder in colmap.

  9. I believe I have all the patches in place to work with 3.4 but getting the following error in feature extractor method:
    uniform sampler2DRect tex; void main(){
    float intensity = dot(vec3(0.299, 0.587, 0.114), texture2DRect(tex,gl_TexCoord[0].xy ).rgb);
    gl_FragColor= vec4(intensity, intensity, intensity, 1.0);}Compile Log
    WARNING: 0:1: ‘sampler2DRect’ : symbol not available in current GLSL version
    ERROR: 0:1: ” : syntax error: #version directive must occur in a shader before anything else
    ERROR: 0:? : ‘pre-mature EOF’ : syntax error syntax error

      1. Thanks for the CUDA hint. I remembered there was a “with CUDA” option and “without CUDA” option on the ColMap download site. Not seeing any instructions on which to choose, I had chosen “without CUDA”. I just went and downloaded “with CUDA” and re-ran. Seems to be working now. Thank you!

  10. Hello, and thank you in advance. I am new to photogrammetry and have tried using your script. As far as I can tell it does the first things with mvs because the folder “output” is created and the images are put in there along with some files, but then it stops. I have set up the path of the binaries for colmap though. I am not sure what to do.

  11. It appears everything ran without error. I looked through all the output and didn’t see anything that looked like one. Now I see 3 files in the model folder: model_dense_mesh_refine_texture.mtl model_dense_mesh_refine_texture.obj model_dense_mesh_refine_texture_material_0_map_Kd.jpg
    What do I do with these? Is there recommended software to view it?

      1. I found that I had Paint3D on my Windows 10 laptop and it was able to open it. However, it asked if I would like to import a texture file. I assumed it meant the .mtl or .jpg file so I browsed to the model folder but it wouldn’t accept either of those files. It finally opened but wasn’t textured. It looked like utter crap. I had used 64 photos of some land that we were thinking of purchasing. I had stood in one spot and took these photos as I gradually turned in a complete circle. I made 3 circles pointing up, out, and down, with at least 50% overlap. This meant some pictures were more distant than others. Is this software only for small objects or can it be used for large land surveys? Apparently not. Can anyone recommend one that works for land surveys?

      2. It should work with photos outside perfectly well; I’ve used it on large dinosaur tracksites. However, there is a knack to taking the photos, and circles pointing out aren’t the best way – you’re better moving around the outside of the area and aiming inward with the cameras.

      3. Taking all of your images from a single position will produce a good panorama, but will not give any stereo (parallax) information that Colmap (or any other 3d system) can work with.

        For this each object (actually each point on the object) that you want to produce a 3d image of needs to have pictures taken of it from several viewpoints. In practice this means you will need to repeat your original photo section with one or more sets of images taken from different positions. These positions should be separated both horizontally and vertically.

        You will probably have to go back several times and take extra photos as you discover areas of your survey which do not have the necessary coverage to produce 3d .

        Alan

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.