Remote repository WIP

[jesterKing]

Hi,

yesterday I've been programming a first prototype of saving and loading data to and from a database (which can reside on a different machine). I started investigating this already a few weeks ago, though. (I guess the lack of an internet connection home 'forced' me to do something actually useful )

I have done this only for limited mesh data now so I can start working on workflow and UI - representation towards the user.

For now I program this using MySQL, but it is sure possible to make this a bit more abstract so other databases could be used as a backend.

I have two screen captures of work until now:

http://www.jester-depot.net/dbblender_004.avi (4.13MB, 49 seconds)

and

http://www.jester-depot.net/dbblender_005.avi (16.9 MB, 2m21s)

Both movies are DivX 5.0.5 encoded.

In the first movie I assign an existing mesh a new name 'db:7' meaning mesh with id 7 from the database, and then refresh (that is: load) the meshdata from the database.

The second movie shows basically the same process, but more elaborate.

In the coming period I'll be adding saving, loading and linking of more data into and from a database. I'll be posting progress in this thread.

/jesterKing

ps. there are also parts 001-003 in the series, but they all suffer bad quality.

[Panther]

Hi jesterKing.

This seems very interesting indeed - And useful !!!

If successful, will this be added to the official BF release ???

Can't wait to try-out a binary of this, please keep us posted on this one - Thanks !!!

[aoe2bug]

This is neat, good idea jesterKing. very insteresting, I hope it is part of blender one day!

[Dani]

Hello! interesting concept.

It kinda makes me think of Verse? Am I wrong? I have 56k so i haven't yet downloaded the movies.

Are your project and Verse incompatible concpet-wise?
Because, since Eskil and Ton seem to see blender's futur in verse, this could be a start... but I may completely off topic since I haven't downloaded those movs. sorry

Anyway! keep it up!

Ciao!
Dani

[aoe2bug]

actually, the idea of having data stored remotely is common to both this and verse, but i think that with verse, the intent is to be able to have multiple people working on, and seeing, media at the same time. whereas this is just accessing data from a server. but then again you could probably do both with both...

I guess eskil or jesterking will explain the differences better, but this is what i understand for now.

[dmoc]

This link may help. I have not read it but it discusses db access in Python in a similar way to PEAR:DB for PHP, ie, db abstraction:

http://www.kitebird.com/articles/pydbapi.html

AFIK VERSE is a network protocol, tailored to exchanging 3D data in realtime so the two could complement each other but in this scenario actual db access would occur on the server and data transported via VERSE. Of course there is still a case for db access from the client to a local db server, say for personal project, which would then later be shifted to a server for group access. So abstraction of db access from Blender would still be desirable.

[jesterKing]

dmoc is right about Verse. But apart from a network protocol, it is also a kind of fileformat in itself.

One of the reasons to explore (mysql) database for repository is that I am not interested in real-time collaboration, wrt to sharing data. Another reason is that I want to retain as much of Blenders way of saving and loading data as possible, so I'm now looking in how to provide hookups to the loading and saving process, so that it still is as close as to how filehandling is done now in Blender.

Atm I am exploring a path where a user can flag individual data for db saving, and for that I have a new movie:

http://www.letwory.net/cvsbuilds/dbblender_008.avi (91s, 10MB, divx 5.0.5).

For this one I have choosen to bring database specific data into a panel so that a user can edit them. This is to allow for saving and loading data to and from different databases.

There's still much work to be done, esp. when I start saving _all_ (linkable) data from ie. a mesh (material, texture, image, etc.). I still have to explore this, but IMO things look promising.

Any feedback on this is very much appreciated. But remember when giving (constructive) feedback, that I try to be as least intrusive as possible into blender, and more importantly, the blender fileformat.

/jesterKing

[Koba]

Hi

Very impressive. I've got two questions:

-Can the database be online? On Blender3d.org, for example?

-Could this system be used to load python scripts?


If the answers are yes and yes then this could be the best thing ever for Blender.

Koba.

[jesterKing]

The answers are yes and yes. I use a local database for testing, but it shouldn't matter where it is situated.

ATM I work only with (limited) mesh data (vertices and faces etc.), but in principle scripts would be saveble and loadable too.

I'm about to showcase saving _and_ loading meshdata. Compiler is doing its best to keep up with me Expect another movie soon.

/jesterKing

[jesterKing]

And the ninth movie in the series:

http://www.letwory.net/cvsbuilds/dbblender_009.avi (150 seconds, 12MB, DivX 5.0.5)

This movie shows both saving and loading of meshdata.

/jesterKing

[jesterKing]

BTW. if anybody is interested in the code (+ progress), you can browse my subversion repository over here: http://www.jester-depot.net/websvn/list ... rev=0&sc=0.

You can also checkout if you like. But write access is prohibited (or any action that modifies the repository in any way).

/jesterKing

[Koba]

Hi

If this thing can do scripts then I think it is one of the most important Blender features ever. So much of Blender's functionality is in the form of Python scripts.

Just imagine it; when someone opens Blender for the first time, if they could see the Blender scripts available with short descriptions, the list would be really impressive! Fibre generation with the Beast, humanoid figure creation with Makehuman, Trees with Lsystem, exporters of all kinds...the list just goes on. Then using your program, they could load the scripts they want off a Blender3d.org database instantly. That would be incredible! No more outdated scripts on the hard drive! Scripts are generally relatively small files so it wouldn't take a second to load even on a slow connection. All you need to add now (as you said) is a nice UI within Blender that gives some info on what is available on the database.

I came up with a similar idea just a few days ago but got a less than luke-warm reception. http://www.blender.org/modules.php?op=m ... highlight=

Could you get this system to place the scripts in the scripts directory and so have the scripts available without file browsing? Maybe it could also replace old versions when an update occurs.

All these possibilities and only thinking about scripts too! I am very excited about this. Keep up the great work!

Koba.

[hypereyes]

Don't let the naysayers get you down Koba.
I think the idea is valuable and well within reasonable. People will always be quick to point out obstacles or tradeoffs and then tell you it just won't work. In my several years in the software industry, I've never taken on a project without having to figure out solutions to problems or without having to decide on a trade-off in design. And if others don't see the value in your idea, then let them have their fun surfing around for scripts and spending their own time determining if there are new tools out there that might help them be more effective.

I say go for it. Start simple. Write a Blender client script that would ping a URL and get back a list or XML doc containing a categorized list of scripts, modify dates, and descriptions. The client would then compare this list against a local XML doc and report any differences which you would then have the option of accepting on a script by script basis.
You could of course then throw in all kinds of options - cache scripts locally, use only best user rated scripts, use only stable versions, always get latest and greatest, etc.

I'd be willing to help out if you'd like - either from a brainstorming perspective or with the implementation.

Good Luck
Brian

[loos]

This thing looks really cool jesterKing! Just a thought about it, couldn't it be used for some sort of integrated content management. Sort of like the way SourceSafe is integrated into Visual Studio? I don't know if any other modeling programs have this, though I have heard there are other tools written externally from modelers to do it (from Alienbrain I think).

Is this kind of direction you were hoping to go in? If not, that's cool too. Just wanted to say the idea looks really neat

loos

[Koba]

Hi

hypereyes - Thanks for your support. I was thinking I could implement a Python script that could do this depending on Python's abilities. Python accesses the internet and gives the user a choice of scripts to download (either updated or not installed). Descriptions are vital to tell the user what does what.

This would probably work via a precisely defined html page of links to scripts with descriptions and version dates. The script would then parse through this to give the user the required options based on a log of the scripts and their versions on the hard drive in the scripts folder. The user's selection would result in the script downloading the scripts sequentially to the scripts folder.

I am unsure of this idea as I am uncertain of Python's internet abilities (I'm only a beginner Python programmer) but if it is feasible, I could work on this in the summer. However, jesterKing's great work here may make this unnecessary depending on how this all turns out. It would be great if jesterKing could include this functionality. The important thing is a list of available scripts not installed on the system with descriptions within Blender to allow the user to decide.

jesterKing - Do you see any of this functionality I am talking about being included into your system? It so it would be great!

Koba