.blend file spec?

General discussion about the development of the open source Blender

Moderators: jesterKing, stiv

matt_e
Posts: 410
Joined: Mon Oct 14, 2002 4:32 am
Location: Sydney, Australia
Contact:

Post by matt_e » Wed Nov 13, 2002 4:12 am

Perhaps an option may be to do what 3DSMax does - use .MAX for its internal format, but also have a format like .3DS for use with exporting/interoperability etc?

dittohead
Posts: 122
Joined: Mon Oct 14, 2002 2:49 am

Post by dittohead » Wed Nov 13, 2002 4:14 am

that's the idea!!

makes more sense.
dittohead

LethalSideParting
Posts: 71
Joined: Mon Oct 21, 2002 12:53 am
Location: Bucks, England

Explanations

Post by LethalSideParting » Wed Nov 13, 2002 2:17 pm

Phew, seems I've started off a small torrent of 'keep it' posts! :oops:
Changing the format seems a bit premature, it would be a quite disruptive change, when the focus is better kept on stabilizing the foundation version. However I do think that splitting out the format reading/writing code into a (shared) library would be a very good idea. This would allow external programs to use blender files easily. Also, language bindings for this library could be created for whichever language people want to use.
Exactly. I probably didn't explain myself too well in my post - sorry. I agree that changing the format right now is a pretty daft thing to do. It'll muck things about right when we don't need it to, and will probably create a whole slew of bugs. However, I was trying to communicate my belief that at the moment, a .blend isn't the easiest thing to write an importer for :wink: , and that this may end up holding back Blender's acceptance in the wider 3D world in the long run. A new easy-to-parse format would be one solution to this, and it was just the only one I could think of at the time. However, a shared library would also be a good way to handle it (and, come to think of it, would allow us to keep the current format whilst allowing other packages to use it...cool...).
Perhaps an option may be to do what 3DSMax does - use .MAX for its internal format, but also have a format like .3DS for use with exporting/interoperability etc?
Now there's a thought....*thinks*....*still thinking*...*can't you just hear the cogs turning?*... The only potential (v small) drawback is that for every scene you want to export, you'd have to save it an a format usable by converters and the like, and then save it in the other package's native format as well (as well as the original .blend or whatever). 3 files for every 1...hmm...how do big studios deal with size issues of something like that, or just don't they mind?
BTW: "not BLEND file" errors are generated when the first 16 bytes in a file are incorrect. Has nothing to do with file size, but happens when saving files with a web browser for example.
Aah!! So that's what is was....Well, that explains about half of my 'not BLEND file' errors, but I do get some of the same error with files that I saved locally, without anything else like a web-browser involved. Bug, maybe...or again just Windoze :D

LethalSideParting

phlip
Posts: 4
Joined: Fri Oct 18, 2002 1:47 pm
Contact:

Post by phlip » Sun Nov 17, 2002 12:28 pm

broken wrote:Perhaps an option may be to do what 3DSMax does - use .MAX for its internal format, but also have a format like .3DS for use with exporting/interoperability etc?
Sounds perfect!
If this was done and the export file format released, I would be happy to write a blend->3DS->DXF->VRML->ETC->ETC->ETC and back file converter for any file formats I can find

pat
Posts: 21
Joined: Mon Nov 18, 2002 3:51 pm
Location: germany

Post by pat » Tue Dec 03, 2002 10:00 am

what about a new file format, :?:
text-based so you could edit it by hand (like vrml), open to everybody so you can write import/export scripts for every 3d-software, supporting mesh, material, UV-tex, ...
would need help from somebody working with 3d-max, maya etc to support everything that is needed to make it a popular 3d-exchange fileformat.
any suggestions :?:
i have a basic idea but i don't know if everything is supported what is used in other 3d suites.
do you think this is a good idea :idea: ?

Vargol
Posts: 8
Joined: Thu Dec 05, 2002 8:14 pm

Re: Explanations

Post by Vargol » Thu Dec 05, 2002 8:27 pm

LethalSideParting wrote:The only potential (v small) drawback is that for every scene you want to export, you'd have to save it an a format usable by converters and the like, and then save it in the other package's native format as well (as well as the original .blend or whatever). 3 files for every 1...hmm...how do big studios deal with size issues of something like that, or just don't they mind?
Well I would hope that Blender would be able to read its own cross platform file format, I would hope the extra format would complete enough to
replace .blend, and perhaps supersced it. Anyway the point is they would only need two files not three if the job is done properly.

Vargol
Posts: 8
Joined: Thu Dec 05, 2002 8:14 pm

Post by Vargol » Thu Dec 05, 2002 8:43 pm

pat wrote:what about a new file format, :?:
text-based so you could edit it by hand (like vrml), open to everybody so you can write import/export scripts for every 3d-software, supporting mesh, material, UV-tex, ...
would need help from somebody working with 3d-max, maya etc to support everything that is needed to make it a popular 3d-exchange fileformat.
any suggestions :?:
i have a basic idea but i don't know if everything is supported what is used in other 3d suites.
do you think this is a good idea :idea: ?
I guess I might as well start the flame wars, but it looks like we are talking XML here.
  • Its human readable, if done properly, also needed for debugging.
    Its extensible, for when blender adds new features.
    No need for every one to write there own file parser, grab your favourite XML parser or Transform the XML.
    Easy to skip nodes that the file converter doesn't understand due to being
    out of date.
    By using a DTD or XMLSchema we can validate files converted too blender format before loading.
Okay...any one whose a XML cynic can now flame away.

theeth
Posts: 500
Joined: Wed Oct 16, 2002 5:47 am
Location: Montreal
Contact:

Post by theeth » Fri Dec 06, 2002 5:27 am

yes, an XML based format would be nice, but that doesn't mean we can't keep the binary based format either.

Also, before starting talking about changing format, maybe one shall look at how the file is saved now (in the code I mean).

Martin
Life is what happens to you when you're busy making other plans.
- John Lennon

pat
Posts: 21
Joined: Mon Nov 18, 2002 3:51 pm
Location: germany

Post by pat » Fri Dec 06, 2002 10:02 am

Vargol wrote:I guess I might as well start the flame wars, but it looks like we are talking XML here.
  • Its human readable, if done properly, also needed for debugging.
    Its extensible, for when blender adds new features.
    No need for every one to write there own file parser, grab your favourite XML parser or Transform the XML.
    Easy to skip nodes that the file converter doesn't understand due to being
    out of date.
    By using a DTD or XMLSchema we can validate files converted too blender format before loading.
Okay...any one whose a XML cynic can now flame away.
sorry, don't know anything about xml
programming skills only basic, a little bit delphi, c++ and python :cry:

Vargol
Posts: 8
Joined: Thu Dec 05, 2002 8:14 pm

Post by Vargol » Fri Dec 06, 2002 7:17 pm

theeth wrote:yes, an XML based format would be nice, but that doesn't mean we can't keep the binary based format either.
I'm all for it being a second format, for easier import/export. If it becomes popular then the blender community might want to consider making it the primary format, with .blend support being then for backwards compatibilty.

theeth wrote: Also, before starting talking about changing format, maybe one shall look at how the file is saved now (in the code I mean).
A brief look at the code (and I admit I may be been looking at the wrong code) suggests its basically a memory dump of the SDNA thing that holds the scene data internally with some headers.

While memory dumps are quick and easy there are lots of problems associated with cross platform data transfer, things like byte reversal, floating point formats, different structure packing. I haven't looked how blender deals with these, or if indeed it attempts to deal with them.

Further to this, memory dumps are not the best starting place for people writing importers/exporters, if requires far to much knowledge of blenders
structs, magic numbers to code / debug etc. Where as....

<sphere>
<radius>5</radius>
<rgba>00AAFF00</rgba>
</sphere>


is pretty much self explainatory, and a DTD (for the none XML savvy a DTD describe what tags are allowed and what order they can come in) gives a good programmer the ability to write a parser with out even looking at blender code which may be necessary in an anti-GPL enviroment.

Dave

theeth
Posts: 500
Joined: Wed Oct 16, 2002 5:47 am
Location: Montreal
Contact:

Post by theeth » Fri Dec 06, 2002 8:24 pm

Ton also highlighted the advantage of using a binary format and the other advantage of the blend format in another post. I'm doing this from memory, so I might be forgetting a couple.

- quick loading quick writing
- multiplatform (a text base format could have problems with linefeed chars)
- smaller memory usage
- backward compatibility achieved by a DNA like structure
- self-repairable

and others that I don't remember.

yes, text based format are nice for importing in other programs, but they are bulky when working with complex scenes which makes them not reallly practical for everyday usage.

Martin
Life is what happens to you when you're busy making other plans.
- John Lennon

Vargol
Posts: 8
Joined: Thu Dec 05, 2002 8:14 pm

Post by Vargol » Sat Dec 07, 2002 4:05 pm

theeth wrote:Ton also highlighted the advantage of using a binary format and the other advantage of the blend format in another post. I'm doing this from memory, so I might be forgetting a couple.


- quick loading quick writing
- smaller memory usage
I can't disagree with that if you use the something like a DOM but event based parsing
can be very fast and lightweight.
- multiplatform (a text base format could have problems with linefeed chars)
We wouldn't have to deal with that the XML parser would, they probably already do. Of course the easy ways around it are to define a line terminator and stick to it (it really only causes a problem due to windows lame /n = /r/n if you write files in text mode) or not use one at all.
However I think what line feed we use is a minor problem compared to endianness, structure packing, native floating point format etc which will become more of a problem the platforms Blender targets.
- backward compatibility achieved by a DNA like structure
- self-repairable
Now I don't understand enough about the SDNA to really comment on this,
but, what about foward compatability, can an importer for a render do a reasonable job one an old file if a comes across a new texture field its never seen before, with XML is easy to skip the node, you know where it ends and where the next node begins you lose that new information but alleast you don't have to bail out.
yes, text based format are nice for importing in other programs, but they are bulky
Ascii scenes tend to compress very nicely, you can get outstanding results on XML with specialised XML compressors. Plus I believe import/export is exactly where this thread was heading.

Dave
Last edited by Vargol on Sat Dec 07, 2002 4:26 pm, edited 1 time in total.

Vargol
Posts: 8
Joined: Thu Dec 05, 2002 8:14 pm

Post by Vargol » Sat Dec 07, 2002 4:15 pm

khkachn wrote:XML would give large file sizes and be much slower to load.
I think you would do better to keep the current .blender files
and adding an XML export with a defined CDC or schema.
I think it would be a bit poor if blender could not import its own export file :? I'd view it as a proof of concept.

Dave

Vargol
Posts: 8
Joined: Thu Dec 05, 2002 8:14 pm

Post by Vargol » Sat Dec 07, 2002 4:29 pm

khkachn wrote:XML would give large file sizes and be much slower to load.
Missed this before. Yes XML can be big, but it compresses very nicely too.
OpenOffice.org's file format is compressed XML and its considerably smaller than the equivalent M$ .doc file, which as it happens is pretty much a memory dump.

Dave

Post Reply