How to edit textures

Background
Many of you may have noticed that there is no reliable information about how to edit texture anywhere to be found on the internet. Space engineers uses the worst image format ever concieved of, .dds, which is a microsoft proprietary image format that cant be opened by the majority of image editing software, includes over 30 different formats with the same suffix, and changes every six months without warning. I am told that photoshop can handle this format but I consider installing adobe products on my machine to be a form of cruelty to both users and machine that is intolerable. My go to program for images is gimp for editing, irfanview for viewing although in the two days it took my to unravel this mess i tried half a dozen recommended programs. PaintNet is recommended by many but there are a number of dds files, both in SE itself and in mods that I found that it can't open.

I hope this guide saves others from the ardous work and allows them to simply create content without going on a murder spree in silicon valley. This guide will probably only work for a few weeks until microsoft changes the format again so I apologise to those in the future that have to start again from scratch.

Getting Started
The only reliable tool I could find for working with .dds files is the official microsoft command line tool.
Install this somewhere on your machine. If you are not familiar with command line tools I apologise but I will be supplying the correct commands below. To open the command prompt on windows press the windows key and R and the same time, and type 'cmd' in. Then press enter.

Converting Existing Textures Into a Format That Doesn't Suck
I am told this process is lossy but really don't care anymore. It certainly doesn't appear to have a major effect.

The command to change an image into a format that you can edit and work with is as follows:
<full path to your texconv foldere>\texconv -nologo -ft png <path to the folder where your texture is>/<filename of the texture>

This will create a png version of the texture in the same folder the original texture was in.
Example command:
C:\DXTex\texconv -nologo -ft png D:\Games\Steam\steamapps\Common\SpaceEngineers\Content\Textures\Voxels\Desert_01_ForAxisY_distance_1_cm.dds

You can change png in the command to bmp, dds, jpg, png, or tiff if you prefer those formats.

Please bear in mind a lot of SE textures have a black alpha channel and will show up as transparent in most image programs. In gimp, if you wish to clear the transparency so you can view and edit the image, use the following steps:
Right click a layer of the image on the layer tab -> select 'add layer mask' -> on the layer mask popup select the 'transfer layer's alpha channel' option. -> right click the layer again -> select 'delete layer mask'.

If you do this you will have to blacken the alpha channel again when you are finished editing (unless you want everything super shiny).

Converting a Format That Doesn't Suck Into a Space Engineers Texture
If you have an image in one of the formats mentioned above and wish to convert it to dds you use the following command:

<full path to your texconv foldere>\texconv -f BC7_UNORM_SRGB -sepalpha <path to the folder where your image is>/<filename of the texture>

Example:
C:\DXTex\DXTex\texconv -f BC7_UNORM_SRGB -sepalpha D:\mySpaceEngineersMod\Textures\myNewTexture.png

Please bear in mind this creates upper case .DDS files which space engineers will not recognise if the XML is looking for a lower case .dds file. You can rename them or change the XML.

You can use BC3_UNORM instead of BC7_UNORM_SRGB as recommended by the KSH guide, this format both works in space engineers and can be opened by normal image programs. Gimp needs the standard dds plugin for this. It is also a good idead to leave off the SRGB part for textures that are not the colour map (*_cm.dds). See the informative post from Harag below for details, and also further discussions at this thread.

This means once you have converted the files once, you can actually edit the texture files directly, without having to export them to a non stupid format. This is not recommended however because you will have to edit all ten of the mipmaps separately by hand. Editing in png or other normal formats and then exporting with the command above will automatically generate all your mipmaps so you only need to concentrate on the main image.

Batch Files
To save time I have created two batch files, ToTIFF.bat asd which will convert all files in a folder to tiff format (chosen because irfanview ignores the transparency on tiff), and ToDDS.bat which converts all images that aren't dds in the current folder to BC7_UNORM_SRGB .DDS files.

These batch files assume that texconv is located in C:\DXTex\texconv, if this is not the case for you you will have to either update the path in the file with a text editor, or move your textconv folder to that location.

 

Thank you for this guide, looks really useful. I am a new modder and have been trying to make a simple plain white texture for the last 48 hours or so with no positive results. Ditto re Keen's terrible documentation - given their project management who can really say what will they'll leave us with as far as docs go. This game seems cursed to be perfectly honest.

 

please link to the download page instead of the download ;-)

 

well, i agree that converting .dds to .png for edit and generating .dds with mipmaps from a .png source is mostly the best way to handle it, but whats the advantage from the command line tool over all the other good programs?

i actually use paint.net to edit and convert .dds to .png and then convert it back with a third party tool thats designed to generate .dds files from other file formats!
i personaly dont like gimp, it loads slower, has a strange working interface and you need to download a lot of plugins befor you even can start with!
so for beginners, that only want a smooth tool to edit textures instead of a powerfull image creator, i can recommend using paint.net ;-)
it loads faster, is smoothly organized and you can start with some powerfull build in features!
and if you still missing something it has a wide community that creates powerfull plugins for it
i cant just recommend to safe in .dds directly with it, it has some strange quality issues!

 

I changed the link to the download page.

The reason I am recommending the command line tool is because it was the only one that reliably worked for all files. I am using files from other modders and both gimp and paintnet were able to open some files and not others.

Using the command line tool with paintnet will work fine, editing tiffs or pngs, but I got really frustrate with all the people recommending paintnet because it simply did not work for me. Is there a plugin or something I need? I can send you a bunch of example files and the error messages paintnet gives me when I try to open them if you dont believe me.

 

afaik you can just use the nvidia dds plugin for photoshop in paint.net to replace paint.net dds exporter.
I'm using Photoshop now but i remember the last time i've used paint.net i exported to dds with nvidias plugin

 

thx ;-)

thats actually a good reason you should mention in the topic ;-)
well, those files cant be opened by the recommended viewer (WTV) aswell, so i just ignore them xP - with irfan view idk, needs to be tested (got deinstalled because it offers to much and is to complicated use for the simple tasks)

theres more than one way to rome ;-)
there are much good tools for several tasks - but theres no "omni-tool" that excels at everything and is comfortable in use too!
more functionality always goes with more or less complexity (depending on the ui)
as stated above, i had just ignored those files - just because i dont need them nor have an idea what they would change if i could use the few dx11 features SE supports xP - even dx9 textures still work well for me
(what a surprise as i noticed the game runs in dx11 mode only^^)

good hint, will try that out^^

 

Added my reason for not using paintNet to the main post.

I also should admit that photoshop is actually a good program, the really bad products from adobe are the other ones, especially acrobat and indesign. But stil I don't like adobe and I do like open source and I dont want to buy or pirate photoshop.

 

well, that is microsoft, apple, google and facebook too!
(all a question on how to use it - there is just no way to force you to use a software product only in the way they want it! - the borders are in your mind and in the code - learn how to break it and youre free )

 

That's because those files use BC7 which isn't known by a lot of programs, blender also included.

 

You are absolutely right. I avoid facebook and apple like the plague, I am very excited about the possibility of linux being viable for games which is slowly becoming true so I can leave microsoft behinf, and well google...
I feel like I cant even pick my nose without google recording every detail of it. If there is an easy way to avoid google products and services I have yet to find it.

 

Visual Studio Community 2013 can open, edit, and save .dds files. I have used that for experiments.

Edit: and it was free, from Microsoft.

 

Good to know, I have 2015 but I will see if it works there and post details if it is useful.

 

That's pretty unqualified vendor bashing. Use whatever you are comfortable with. I prefer to be fast and have immediate 3D feedback. Neither Gimp nor Paint.net nor Photoshop on its own give me that. I use Allegorithmic's Substance tools, Quixel Suite also works well from what I hear (the later being a texturing plugin for Photoshop).

The project has moved to GitHub. Current releases are here.

It is a very good decision to use .dds because DirectX mandates that GPUs be able to decompress this format on-the-fly in hardware. This means you save ~75% of texture memory compared to other formats because they need to be decompressed before storing them in GPU memory.
Also, as a texturing-artist you need to be aware which subformat serves which purpose. As a rule of thumb for DirectX11 engines: use BC6 for HDR images like environment maps and image-based lighting and BC7 for everything else because older compression formats have less quality and are no faster to decompress.

This is misinformation. As far as I know using the _SRGB formats controls automatic colorspace conversion. It's a good idea to use that for the _cm textures to avoid color banding in dark areas of a texture (that's the reason why sRGB exists). The other textures should be authored in linear space (no _SRGB suffix) because they don't contain color-information. You also need to be aware which colorspace your editing software is working and saving images in. Also see my post here.

Why? On-the-fly hardware decompression and thus more GPU memory for other things is a good thing!

 

Thanks Harag for clarifying a lot of things about tools.

In response to your other comments, I obviously know a lot less than you about texture and image technology so your input is valued. I was just frustrated with the limited information available and wanted to post what I had found out. Here are some more specific notes:

I did not mean to imply I thought it was a bad decision to use .dds, I was under the impression that there really isn't much choice in this kind of application. I said that .dds is a bad format. My issues with it are not technical, I find that it is badly documented and confusing to use. Most of this is microsoft's fault and this applies to many file types they have created, most notably their document formats. I think it would be sensible to split dds into different suffixes, especially when old file formats are obsoleted due to new DX versions. The amount of time it took me to find a simple reference that told me what DXGI_FORMAT enumerable members correspond to which DXT and BC versions (the terminology used by image programs to describe what is supported) seemed unreasonable to me, given how basic the information is. The number of image applications that claim to support a certain standard (including all the ones recommended on these forums except maybe photoshop) but in fact only support a limited subset of these is not really acceptable. Obviously the application developers themselves share a lot of blame for this but microsoft hardly makes it easy on them. Basically it is the difference between maintaining a software monopoly and maintaining good, usable open standards.

Could you please elaborate on this some more? I will change the original post but I would like to understand more about it if you have the time to spare. The post you link to was one I saw while figuring all this out and was a real breakthrough, one of the most useful on the whole forum on this subject. The only reason I was using BC3 without colorspace was so it could open in more applications. Given that I am editing tiffs anyway (you use .tga, any specific reason for the preference?), if using the command lines you recommend improve the quality and compression in game then I will change to that.

I also reserve the right to indulge in unqualified vendor bashing, just as the vendors reserve the right to make their customers' lives miserable.

 

Bear in mind that I'm no expert in this either.

In shader code colors are usually handled as normalized floating point numbers (i.e. in the range [0.0,1.0]) and used in a linear manner (i.e. an intensity of 0.6 is twice as bright as 0.3). This just makes the math easier. On disk colors are stored as unsigned integer numbers in the range [0,255] because this takes up less space. So when accessing the image-data from a shader a conversion needs to happen. This is the meaning of the _UNORM format-modifier.

But this is not the only conversion that should happen. The human eye doesn't discern low and high light-levels equally well. It's good at low levels and not as good at high levels. So distributing light levels equally over the range [0,255] results in the problem that you don't have enough numbers (you lack resolution) to represent all the low levels the human eye can discern - which means the eye will notice that there is a gap in-between one light-level and the next. Luckily you have more numbers for the high levels than you need. So it's a good idea to distribute the light-levels in a non-linear fashion. This is usually called Gamma correction. Among the available distributions sRGB is the one that is most widely used and with the _SRGB modifier you tell DirectX to handle that conversion for you. Note that most image editing software also works in sRGB by default (for example in Gimp you can checke via Image > Mode > Assign ICC Color Profile).

Keep in mind that sRGB is only useful for color-data that is stored as integers. Other data like roughness or normal maps should be stored linearly because human perception is not involved here so it makes no sense to have an uneven distribution. Also, if your data is stored as floating point you don't suffer from the resolution problem because it's in the nature of floating point numbers to have higher resolution for small values.

Microsoft actually has a lot of this documented:

• DXGI_FORMAT modifiers
• Data Conversion Rules
  
• Texture Block Compression in DirectX11
• Block Compression (Direct3D 10)

And as I've already said, use the best block compression format available with texconv. Not being able to open the resulting .dds file in Gimp (or Photoshop) shouldn't be a consideration because block-compressed .dds is meant for the engine and trades image fidelity for a reduced texture size (i.e. it's a lossy compression and BC7 is not as lossy as BC3). Your source material should be kept in another image format that is as lossless as possible (.exr, .xcf, .psd, whatever). If you want to preview the texture before putting it into the engine use a viewer that can read BC7, like PicoPixel.

Maybe this is also useful:

• John Hable, Uncharted 2 HDR lighting

 

Thanks a lot. I have changed that information in the first post. There are a couple of other things though. You aren't using the -m command for mipmaps. When I omitted that I got textures that were black unless viewed from very close. Can you shed some light on this? Were you doing the mipmaps manually? You are also setting texconv to use directx 9. Is this just because the post is older and the game used to support that? Should we be forcing that to 11 now?

 

According to texconv's manpage it calculates the full mipmap pyramid if you don't use the -m option. It's only useful to limit the number of mipmap levels, say to 1 if you don't want any levels. I'm unsure what's causing you problems.

And yes, the _de and _ns textures are for the DirectX9 renderer of the game which is no longer developed as of March 2016. So you can completely omit those two if you don't intend to author textures for it.

 

Ok thanks. I will test the mipmap thing again, it is possible I was doing something wrong or stupid. Then I will upload a new .bat file, those files save a lot of time if you can just make a texture edit folder and pack everything in there to modify.

 

that thread has changed my mind about texturing, thx for all the info guys
have reported it to get sticky ;-)

 

So i updated the ToDDS.bat file. I have been having so many other problems with modding (turns out the texture guide is not the only one that is totally obsolete and misleading) that I still haven't got around to testing it out to be sure it is better. But I trust Harag who seems to have a handle on all of this, so I followed his advice.

Please post here if there are any issues with using the file, I would hate for this to be another of these guide that wastes people's time without helping.

 

Its also possible to convert textures to TGA and back, which is more comfortable to use since it doesn't "lose" the alpha channel to transparency, like Tiff and PNG do.

 

Harag, over the last 24 months as I have gradually gone from having never modded and never used anything above MS Paint, continually when there's a topic I've struggled to understand through online reading etc, it's often you that actually manages to make it understandable.

It's definitely worth mentioning Alegorithmics products as Harag said. I've had Substance Designer for a year or more and it's a very cool tool for building textures and with a bit of effort being able to perform massive sweeping non-destructive changes. Last week I finally bought Substance Painter on a whim, and man I wish I'd done it long, long ago. The quality of my texturing has gone up 10 fold almost over night, especially being able to paint height details directly onto a mesh. Got some big mod updates coming through that will see a massive quality improvement.

 

I wanted to add these tools still work and this is the first time I've been able to successfully load an SE texture into Photoshop.

Also for Photoshop to separate the alpha you select the layer and then from the Layer menu select Layer Mask -> From Transparency. Then you can disable the layer mask to see the texture correctly.