Raster Blaster is a QGIS plugin I put together to help speed up the creation of GeoTIFFs and Cloud-Optimised GeoTIFFs (COGs). This post goes into the backstory of the plugin and some of the basics of using it.
The single most time consuming part of adding content to this website is the process of georeferencing, the process of aligning an image or a map to real world coordinates so it appears in the correct geographical location when loaded into software such as QGIS or uploaded to a website like Soar.
In my process of georeferencing there are two main parts to this process. Placing the Ground Control Points (GCPs) and the software, in this case QGIS, processing the file. In this post I'm going to focus on the second part, the processing.
QGIS is an incredible free and open source tool, I couldn't do what I do without it. However, where it falls short for my work is georeferencing large raster (image) files. To give you an idea of how large I'm talking about, many images on this site can be around one or two GB, but some of the images on this website are nearly 300,000 pixels by 100,000 pixels and can be several, or even dozens of GB (and they're not even in colour!!).
So after placing somewhere between 50-200 GCPs, I tell QGIS to start processing the file into a georeferenced format known as a GeoTIFF. Depending on the size this process can take an hour to several hours. I have a pretty powerful PC with SSD storage, lots of RAM and a good CPU, but as I understand it (please correct me if I'm wrong), the default QGIS georeferencing tool only utilises a single CPU core. In my case, this means that I have 11 CPU cores sitting around doing 'nothing' when they could be used to help speed up this process.
I'm not a 'code-y' person, if you will (please keep this in mind as I attempt to explain technical things below, I'm trying to learn). During COVID lockdown I tried to learn how to code but it was overwhelming to me. I'm a very visual person who doesn't feel super comfortable writing things unless it's purely an off the top of the head kind of thing (like this post), but I kept hearing about how I could use something called GDAL to speed up my processing.
Finally, in April of this year I reached a breaking point. I needed to speed up the process instead of waiting hours at a time for a single image to process. I scoured the internet for how to do this but everything felt so technical. Command line interfaces, scripts, all of that. Which is great, don't get me wrong! But I surely can't be the only one who struggles with this, so I decided to try and come up with a solution. I didn't really know who to ask about this, and I couldn't afford to hire anyone, so I went to ChatGPT... I know, but wait. I'm not someone who runs their life on 'AI', or claims it's our saviour, but I thought I'd try it and see if it could help me wrap my head around this, maybe even put together a product for people to use.
After several weeks of wrestling and being gaslit by the glorified Large Language Model, wondering why it was trying to do something completely different to what I asked, I actually have something. Raster Blaster (intended to sound like the raster is speedy, not being blown up, I hope!) A plugin that (at least of my system) is stable, extremely fast, and easy for anyone to use. I want to share this with you. I want the community to help make it better.
Raster Blaster uses the power of GDAL to take advantage of more of your system resources than standard QGIS would. Specifically with the following lines of code which use all CPU cores and threads.
"--config", "GDAL_NUM_THREADS", "ALL_CPUS",
"-wo", "NUM_THREADS=ALL_CPUS",
Want to know how much of a difference this can make? The graph below is just one example of the power of this plugin.
.png)
Now of course, the actual speed comes down to the hardware inside of the computer you are using, but I think this pretty clearly shows that it's faster. And to anyone who works with large raster, that's pretty good!
Okay, but how do you use it? I've tried to make it easy.
When you install the plugin and open QGIS you will have two ways to access the plugin's features. Option 1: Go to the QGIS toolbar > Plugins > Raster Blaster. Option 2: Go to the QGIS toolbar > Layers > Georeferencer > The buttons will be in the Georeferencer's toolbar.
You'll see there are three buttons:
Points→GeoTIFF reads your GCP .points file and creates a GeoTIFF.
Points→COG does the same thing as Points→GeoTIFF but creates a Cloud-Optimised GeoTIFF (COG).
And lastly, GeoTIFF→COG converts an existing GeoTIFF into a COG.
I really wanted to make use of the COG format, especially with JPEG compression, as I've fallen in love with it. From now on, all of my images uploaded to Soar, which are then embedded on this site, will be COGs.
Another goal of mine was to try and make this as easy to use as possible. As you can see in the image below the dialog boxes are pretty easy to use. Either one or two file inputs, a few options, and a single output.
Now, one thing to note is that when you hit the 'Run' button, QGIS does tend to lock up, but you should see the OSGeo4W window open. Just let it sit. This means its running. Once the process is complete QGIS will become responsive again and deliver a message about how long the process took.
It's important to keep in mind that as of the writing of this post the plugin is very experimental, mostly written by ChatGPT (😳), and may still have issues that I haven't found yet.
You can download Raster Blaster for free on GitHub: https://github.com/harryspacefromspace/rasterblaster
Also directly from the QGIS Plugins page: https://plugins.qgis.org/plugins/rasterblaster/
