ividyon will never get this done, will he.

An Ideal Remedy for 3ds2unr

For questions and discussion about UnrealEd, UnrealScript, and other aspects of Unreal Engine design.

Moderators: ividyon, Semfry, zYnthetic

User avatar Jet v4.3.5
Skaarj Elder Skaarj Elder
Posts: 1239
Joined: 24 Dec 2007, 17:40
Contact:

Subject: An Ideal Remedy for 3ds2unr

Post Posted: 27 Mar 2016, 03:39

I want to know what people think would be the most suitable workflow for converting their animated meshes (in Blender, Max, Maya, Wings3D, etc.) through a single middle-man application. If you could take the goal that 3ds2unr was built to accomplish, and replace its messy brokenness with something more functional and convenient, how might it work?

I'm thinking along these lines:
    =========================================================================
  • File Format
    What file format(s) would be most appropriate? Let's...just accept that having tried .3ds already, we omit that from the possible formats. It's terrible for interchange, and its structure is not completely documented or easy to parse.

    I've looked at the .obj file format, and its structure, aside from parsing its human-readable format, seems to be a perfect fit. I'm assuming this will be good for everybody in some fashion or another.

    If there are any other formats, or deep opposition to the .obj format, make a case.

    =========================================================================
  • Conversion Prep
    When preparing files for conversion, are there ways people would like them to be organized before the conversion?

    Personally, I want whatever it is to allow almost complete automation, without too much mandatory command line fiddling in situations that really don't need it. If you start the program, it finishes, and you have a perfect working set of files (_d.3d, _a.3d, .uc), that's what I would want. The question is what might be the best way to set up our frame files?

    At least by default, I'm thinking a folder hierarchy would be the most user friendly. The program sits in the package's main folder. Inside of this folder, there are "m_ModelName" folders (template: "m_*"), and within these are a_AnimSeqName folders (template: "a_*"). In these are your frame models, under some numbering system; Blender, as an example, exports its .obj files in the format "AnimName_nnnnnn.obj" where the "nnnnnn" is a frame number (such as "Firing_000006.obj"). The program would import each model defined with the m_ModelName folders, and each animation sequence for these models with the a_AnimSeqName folders.

    This method seems to make sense for the modeling program I use, but I'm curious if this sounds completely weird or won't work for others. For instance, if you are organizing all animation frames as individual objects within one file, and using an internal naming and numbering convention instead of individual frame files, then an alternative would be much more convenient. If you want, please provide some example .obj files to give an idea of how you organize your animations.

    =========================================================================
  • User Interface
    Any thoughts on a user interface. A good GUI program, or using a good .ini system for configuration and sticking to the command prompt interface. Model (and surface drawing) preview windows, stats, etc.

    The .ini seems like a great compromise for me. For the file staging mentioned in the points above, one could theoretically set up the converter with settings in an .ini to meet their needs, and the converter would accommodate. One could even imagine having specific presets to remove the need for excess fiddling, and have a "CUSTOM" preset for when people want to get technical.

    The .ini method is a fairly easy way to do things for both the programmer and the user, where a fully-featured GUI would be a bit more difficult to implement, but prettier and possibly easier for a user standpoint.

Any other comments, concerns, or ideas that are applicable would be great.
Last edited by Jet v4.3.5 on 03 Apr 2016, 22:29, edited 3 times in total.
Image
ModDb Portfolio
"Bear," she cried. "I love you. Pull my head off."

User avatar AlCapowned
Skaarj Warlord Skaarj Warlord
Posts: 861
Joined: 19 Dec 2009, 22:25

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 27 Mar 2016, 04:33

I would love to see an alternative to 3ds2unr, and your plan sounds like it would be sooo much better than u3d.

The OBJ format is good. The folder hierarchy option doesn't sound too bad, but with 3ds Max I either export all of the frames for an animation into one file, or export all of the frames for all of the animations into one file. I never did look to see if it can export a bunch of individual frames easily.

While a GUI would be nice, I would be perfectly fine with using an ini and the command prompt.

Being able to generate a class file with all of the animations listed (like 3ds2unr does) would be good, and so would the ability to append frames. A scaling option to decrease the mesh jiggling problem or fix models that go over the 128x128x128 box would be downright luxurious.

Very exciting stuff!

User avatar Jet v4.3.5
Skaarj Elder Skaarj Elder
Posts: 1239
Joined: 24 Dec 2007, 17:40
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 27 Mar 2016, 06:09

AlCapowned wrote:I would love to see an alternative to 3ds2unr, and your plan sounds like it would be sooo much better than u3d.

The OBJ format is good. The folder hierarchy option doesn't sound too bad, but with 3ds Max I either export all of the frames for an animation into one file, or export all of the frames for all of the animations into one file. I never did look to see if it can export a bunch of individual frames easily.

While a GUI would be nice, I would be perfectly fine with using an ini and the command prompt.

Being able to generate a class file with all of the animations listed (like 3ds2unr does) would be good, and so would the ability to append frames. A scaling option to decrease the mesh jiggling problem or fix models that go over the 128x128x128 box would be downright luxurious.

Very exciting stuff!


That's helpful, thanks! I just edited the main thread a bit, but to repeat it here: please feel free to provide good example .obj files. From what I can tell, they come in many different shapes and sizes based on how you organize things within the modeling application that you export it from. So any examples could be invaluable, and it would dictate at least one other standard and straightforward technique alongside the folder hierarchy option.

Definitely assume that a .uc class file will be generated. I wouldn't want it any other way.

The appending is a good point. It'll take some extra analysis to get right, but in the meantime I wonder if there's a good explanation of how it works, or documentation, or if you could explain it yourself? I may just need to read the 3ds2unr reference material.

I'm very concerned about the automatic scaling being available (even as a .ini option). It's been one of 3ds2unr's biggest limitations. I also wonder if it would be useful to have an option to shift the model as well? I'm essentially imagining this as a way to shift the whole model so that the bounding box encompassing all vertices of all frames is centered on the origin. If the automatic scaling is enabled as well, this would allow the model to reach its maximum size within the limited bounds for Unreal's vertex meshes. So if you want to preserve the relative position of your model and simply scale it, that's one option, but you can also slide the mesh around and THEN scale it to further minimize the bubbly vertex effect.
Image
ModDb Portfolio
"Bear," she cried. "I love you. Pull my head off."

User avatar ebd
Trustee Member Trustee Member
Posts: 428
Joined: 05 Apr 2008, 19:08
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 27 Mar 2016, 19:40

Re-write 3ds2unr as a python script for blender that exports using the animation timeline from blender directly to unreal's format with a parameterized frames per second, skipping .3ds entirely. Export multiple animations at once and then split them up in the generated .uc file.

After peeking at the source code the structure of _a and _d files is actually far simpler than I thought they would be.

I'd personally prefer something with wings3d of course, but it'd be a little more involved (wings is just a modelling tool; it has no support for animation)

User avatar Jet v4.3.5
Skaarj Elder Skaarj Elder
Posts: 1239
Joined: 24 Dec 2007, 17:40
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 27 Mar 2016, 23:39

ebd wrote:Re-write 3ds2unr as a python script for blender that exports using the animation timeline from blender directly to unreal's format with a parameterized frames per second, skipping .3ds entirely. Export multiple animations at once and then split them up in the generated .uc file.

After peeking at the source code the structure of _a and _d files is actually far simpler than I thought they would be.

I'd personally prefer something with wings3d of course, but it'd be a little more involved (wings is just a modelling tool; it has no support for animation)


That's an idea I'd want to look into eventually, after a standardized tool was produced. I'd need to look into Blender's Python API, and any data size constraints on writing the _d.3d and _a.3d files will require some special treatment since Python has such a loose implicit type system.

But for the moment, the design we seem to be going for I'm terms of the standalone converter should work, just with the extra export step in Blender. And Blender's frame exporting for .obj works essentially how you describe. Then you have the files in the same form that I described in the main post, and can easily push them through the converter. A custom exporter for a given application would, of course, be killer.
Image
ModDb Portfolio
"Bear," she cried. "I love you. Pull my head off."

User avatar Jet v4.3.5
Skaarj Elder Skaarj Elder
Posts: 1239
Joined: 24 Dec 2007, 17:40
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 29 Mar 2016, 20:59

So my first goal for this little project was to ensure that the interfaces from 3ds2unr's source code were reliable, so that I could use them later without worrying if something was going to break for no reason. My first concern was single-frame model construction, so that I could reliably construct first a triangle and then a cube, each with correct UV maps, as a starting point, and I've just finished getting both to work:




The ugly skin you see was made for the cube to track correct edges and reconstruction, and at present things are pretty much perfectly aligned. I just threw it on the triangle for testing purposes. At the moment I've been using methods for objects already present in the 3ds2unr code, and modifying them to work generically on more compilers while using appropriate data sizes, among other things. The coordinates for vertices and their UVs have so far been hand set within the code, but eventually I'll get to parsing .obj files so I don't have to do the math and typing myself.

Overall, getting those two models in is good progress. Also remember: any .obj files and descriptions of how you set up your objects and materials/textures before export is invaluable information for getting this to work cleanly. I'll have a better spec together and I can accommodate your way of model organization more easily if I can see what people are most likely to throw at the converter once it has more functionality.
Image
ModDb Portfolio
"Bear," she cried. "I love you. Pull my head off."

User avatar AlCapowned
Skaarj Warlord Skaarj Warlord
Posts: 861
Joined: 19 Dec 2009, 22:25

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 05 Apr 2016, 16:35

Here's an example .obj file: http://www.mediafire.com/download/wo1sayog31cboyr/objexample.zip

I don't remember if it has the materials set up, but I usually use the Skin## format for the names.

Edit: For some reason MediaFire is doing this sharing protection thing. Not sure if the link will work or not.

User avatar Jet v4.3.5
Skaarj Elder Skaarj Elder
Posts: 1239
Joined: 24 Dec 2007, 17:40
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 05 Apr 2016, 21:03

Thanks for that! If you open a .obj file in a text editor, you'll see fairly readable stuff describing the model. Yours looked about how I expected it to, and since I'll be parsing that information to get as concrete of data as possible, it's good that it's consistent. Extraordinarily helpful.

I see that this is a single-animation model. If you made more animations, would you do something with the frame object names such as "Jumping000, Jumping001" and "Running000, Running001, Running002" as an example?
Image
ModDb Portfolio
"Bear," she cried. "I love you. Pull my head off."

User avatar AlCapowned
Skaarj Warlord Skaarj Warlord
Posts: 861
Joined: 19 Dec 2009, 22:25

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 06 Apr 2016, 01:43

I keep the names to OBJ## for each frame in the 3ds file, but only because that's what u3d and 3ds2unr need. If given the option, I wouldn't mind having different frame names for each animation. It would certainly make keeping track of everything easier when I mess with the scale in Max.

User avatar Gustavo6046
Skaarj Scout Skaarj Scout
Posts: 20
Joined: 09 Apr 2016, 14:35
Location: Porto Alegre, Rio Grande do Sul, Brazil
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 09 Apr 2016, 14:43

There could be a command-line option so that it could also parse a single .obj file with the animations (for example, "gun1" frames 0 to 12 are fire and 12 to 20 are alt-fire and 20 to 25 are down, 25 to 30 are up and 30 to 41 are cocking (that idle animation?)).

In a side note, why isn't Blender exporting animated models to .3ds or .obj?

Finally, sorry for the bump, but I though this a excellent idea and wanted to do my feedback. :^)

EDIT: I can do a .obj file with what I mean, but I'm not right now.

EDIT 2: May I take a shot at Python at looking at the structure of .obj? I will be able to parse .obj files (but not convert to .3d :( )
This is the 700th member!

Half-veteran modeller. Sucker at gaming. Quite nice at coding. O.K. at texturing. Good at mapping in general.

What else do you need? :)

User avatar Jet v4.3.5
Skaarj Elder Skaarj Elder
Posts: 1239
Joined: 24 Dec 2007, 17:40
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 11 Apr 2016, 00:23

Gustavo6046 wrote:There could be a command-line option so that it could also parse a single .obj file with the animations (for example, "gun1" frames 0 to 12 are fire and 12 to 20 are alt-fire and 20 to 25 are down, 25 to 30 are up and 30 to 41 are cocking (that idle animation?)).

In a side note, why isn't Blender exporting animated models to .3ds or .obj?

Finally, sorry for the bump, but I though this a excellent idea and wanted to do my feedback. :^)

EDIT: I can do a .obj file with what I mean, but I'm not right now.

EDIT 2: May I take a shot at Python at looking at the structure of .obj? I will be able to parse .obj files (but not convert to .3d :( )


Well, how the command line comes into play is still pending. I've been working on some tests, prototype code, etc. before I really decide what I can and cannot set in my spec. As discussed in earlier posts, a configuration file of some kind might hold a user's preferences so that the converter can run without the user screwing around with the command line each time. However, if there's a good reason to use it, I will.

So if you want to load a single .obj file full of animation sequences and frames, or if you want to deal with the folder scanning method, you can set that in the config file and run the executable every time afterwards with the expected results. And any other variables of conversion that might be preserved would sit in the config file as well.

As far as the .obj and .3ds exporting from Blender, I'm certain it exports both. But because the .3ds format is so esoteric and shakily documented, it's no wonder that Blender's exporter somehow exports .3ds files in a way that 3ds2unr doesn't like. Not mentioning how unsatisfying 3ds2unr is from a usability standpoint, it's the main reason I see value in a new converter that supports .obj.

Now for Blender, someone writing an exporter would need 1) an understanding of the Blender API for Python programmers (so you can fetch geometry info, selected objects, etc.), and 2) the capability of manipulating and writing to files data in a binary sense, all in Python. I'm not familiar with either (only Python fundamentals), but please feel free to try yourself. If you get it working, then that would be fantastic!

Look at this page for info on the .3d files, and this page for info on .obj files.
Image
ModDb Portfolio
"Bear," she cried. "I love you. Pull my head off."

User avatar Gustavo6046
Skaarj Scout Skaarj Scout
Posts: 20
Joined: 09 Apr 2016, 14:35
Location: Porto Alegre, Rio Grande do Sul, Brazil
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 11 Apr 2016, 14:16

Jet v4.3.5 wrote:Look at this page for info on the .3d files, and this page for info on .obj files.


Yes, that might be the format, but I can't seem to find a pattern simply by looking at a binary data file at XVI32.

I mean, how do we transcribe these ints, shorts, floats, and whatnot's into the binary format?

Same applies for most file format websites that doesn't describe data as "bytes".

EDIT: The plan is as follows:

1) Write the parser that will parse .obj files.
2) Write how the data will be stored in RAM.
3) Write how the data will be converted into 3D format and written into two files; aniv and data files.
This is the 700th member!

Half-veteran modeller. Sucker at gaming. Quite nice at coding. O.K. at texturing. Good at mapping in general.

What else do you need? :)

User avatar ebd
Trustee Member Trustee Member
Posts: 428
Joined: 05 Apr 2008, 19:08
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 12 Apr 2016, 03:14

Gustavo6046 wrote:
Jet v4.3.5 wrote:Look at this page for info on the .3d files, and this page for info on .obj files.


Yes, that might be the format, but I can't seem to find a pattern simply by looking at a binary data file at XVI32.

I mean, how do we transcribe these ints, shorts, floats, and whatnot's into the binary format?

Same applies for most file format websites that doesn't describe data as "bytes".

EDIT: The plan is as follows:

1) Write the parser that will parse .obj files.
2) Write how the data will be stored in RAM.
3) Write how the data will be converted into 3D format and written into two files; aniv and data files.
UnrealModel.cpp from the 3ds2unr source describes pretty clearly how they are writing the binary files.

► Show Spoiler

User avatar Gustavo6046
Skaarj Scout Skaarj Scout
Posts: 20
Joined: 09 Apr 2016, 14:35
Location: Porto Alegre, Rio Grande do Sul, Brazil
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 12 Apr 2016, 18:05

ebd wrote:
Gustavo6046 wrote:
Jet v4.3.5 wrote:Look at this page for info on the .3d files, and this page for info on .obj files.


Yes, that might be the format, but I can't seem to find a pattern simply by looking at a binary data file at XVI32.

I mean, how do we transcribe these ints, shorts, floats, and whatnot's into the binary format?

Same applies for most file format websites that doesn't describe data as "bytes".

EDIT: The plan is as follows:

1) Write the parser that will parse .obj files.
2) Write how the data will be stored in RAM.
3) Write how the data will be converted into 3D format and written into two files; aniv and data files.
UnrealModel.cpp from the 3ds2unr source describes pretty clearly how they are writing the binary files.

► Show Spoiler



Hnnnng C++! But unfortunately I can't really understand the code, and the Code::Blocks compiler is giving me a lot of errors.

It would maybe be easier as pseudo-code or maybe Python?

Also, the code lacks many classes that are vital, e.g. cUnrealPolygon.
This is the 700th member!

Half-veteran modeller. Sucker at gaming. Quite nice at coding. O.K. at texturing. Good at mapping in general.

What else do you need? :)

User avatar Jet v4.3.5
Skaarj Elder Skaarj Elder
Posts: 1239
Joined: 24 Dec 2007, 17:40
Contact:

Subject: Re: An Ideal Remedy for 3ds2unr

Post Posted: 12 Apr 2016, 18:56

Gustavo6046 wrote:Hnnnng C++! But unfortunately I can't really understand the code, and the Code::Blocks compiler is giving me a lot of errors.

It would maybe be easier as pseudo-code or maybe Python?

Also, the code lacks many classes that are vital, e.g. cUnrealPolygon.


Look, if you needed anything more than what ebd posted, or what is mentioned in the documentation I linked earlier, it's that the data types on your system (such as ints, longs, shorts, etc.) may be a different bit width than what is necessary for the .3d files.

To solve that, include this header in your C/C++ code:

Code: Select all

#include <stdint.h>


Then these substitutions should probably work if you're using a modern machine:

Signed:
  • char = int8_t
  • short int = int16_t
  • int = int32_t
  • long int = int32_t
  • long long int = int64_t
Unsigned:
  • unsigned char = uint8_t
  • unsigned short int = uint16_t
  • unsigned int = uint32_t
  • unsigned long int = uint32_t
  • unsigned long long int = uint64_t

You can also use those definitions to intuit bit width sizes for other languages if they support them. I know Python has the library "ctypes", which might help you if that's your language of choice.

At this point you have more information than I did starting out. Otherwise, if the code doesn't make sense to you, study it and write notes until it does. This is (fundamental) computer science, and unless you're paying someone else to teach you this stuff, there's a limit to what understanding people can just give you. It's ultimately your sole responsibility to build your own skills, even under the guidance of others.

EDIT: One last thing that may help you - the earliest public version of 3ds2unr has the original author's source code included with the tool. 3ds2unr was later maintained by other Legend employees, and the source code for the only other realease, version 119, was never released.
Image
ModDb Portfolio
"Bear," she cried. "I love you. Pull my head off."

Next

Who is online

Users browsing this forum: No registered users and 2 guests