Blender Python Consortium: Initial proposal

Scripting in Blender with Python, and working on the API

Moderators: jesterKing, stiv

Post Reply
reD_Fox
Posts: 0
Joined: Sat Mar 12, 2005 2:56 pm
Location: Abilene, KS

Blender Python Consortium: Initial proposal

Post by reD_Fox »

Proposal for the "Blender Python Consortium" (BPyC)

note: this topic has been posted at elysiun as well
http://www.elysiun.com/forum/viewtopic.php?t=46772

This proposal is created mainly because of the following circumstances:

1. Disarray. Currently, Python scripts created for Blender are scattered across forums, authors' websites, and other web (and non-web) locations. There is no central location for retrieving even half of these scripts. This leads to...

2. Circulation of out-of-date versions. Often, links to certain scripts from web sites will be out of date, or will be hosting a version of a script that has since been updated by the author.

3. Duplication of effort. Without an organized system, many scripts are created "twice" because one of the authors was not aware that a script had already been created to fulfill a particular need.

4. Fragmented authorship. Very few scripts have been created as a "team effort." Most script authors work (and struggle) on small, fragmented scripts, or are working on scripts that could be benefited by a multi-person team.

5. Limited script inclusion with the Blender releases. The official Blender release cannot and should not include every single script. The download would be too large, and the maintenance overwhelming. Not only that, but it is often hard for the authors to "perfect" their scripts for release status.

6. Lack of synchronization. There have been some efforts to help users keep their "official release scripts" up to date. However, there is no real mechanism in place to help users stay up to date with any of the other scripts they might have chose to add to their collection.


To help combat these (and possibly other) problems with creating and publishing Python scripts for Blender, I propose the founding of the "Blender Python Consortium" (BPyC). An alternative name may be assigned (or none at all, for that matter), but for this proposal I will continue to refer to it by that title. In its most basic form, it must consist of the following elements:

1. The committee. The committee will need to be a specially qualified team of people experienced in Python (and other) programming, and familiar with Blender's workflow and development.

2. The database. A major part of the BPyC will be the central storage of a large number of scripts. This will need to be organized and accessed via a web-based interface.

3. The authors. The committee will be real people with real lives, and cannot be tied down with manually maintaining and selecting scripts. For this reason, a major part of the BPyC will need to be the script developers themselves. The authors of the scripts will be the "work force" that ensures that the BPyC succeeds.

4. Web developers. Although these will not necessarily be a recognized element of the BPyC, its web-based storage and interface demands extensive website design and maintenance.


To best organize this initial proposal, I will present the functions of the BPyC from the perspective of three groups of people.

1. The committee. The committee's view of the BPyC is top-level. They are ultimately the ones responsible for any decisions that need to be made in the BPyC. It should also be their responsibility to publish and maintain a small list of Blender related Python helps, maintain the major website content, create a list of guidelines or standards for scripts, monitor script activity within the database, offer constructive comments to the authors, and maintain "The Core" and other organized packages (more on this a little later). I will elaborate on each point a bit more.
- The publishing of a "Python helps list" could be links to tutorials, books, or documentation, or could be actual documentation created by the committee specifically for Blender-Python scripting. Regardless, its purpose is to provide a brief, yet helpful, source for people looking to improve their Blender-Python skills.
- The maintenance of the website content may be allotted to individuals other than the committee. However, the committee should play a central role in the maintenance of the "major" parts of the website content.
- A list of guidelines or standards created by the committee will not be "hard rules that you must follow," but rather suggestions for best coding practices within Blender. They may be something like "It is recommended that each script contain a menu header in the following form..." Or, "It is recommended that every script created for the 'mesh' category check the edit-mode status of the current mesh..."
- The committee should also monitor the additions and updates to the script database. If some joker decides to post a four-line Blender-Python "hello world" script, the committee should take necessary action to ensure that the script is not added to the database. They can also communicate with the authors to recommend changes to a script. This could be anything from recommending better coding techniques to explaining that a similar script already exists, and that the author may want to consider teaming up with the previous author to add the wanted functionality.
- Since the BPyC will seek to serve as a central repository for all Python scripts, it is necessary that some structure be instituted to separate the "good" scripts from the "bad." For this reason, I suggest the maintenance of a "core" package synchronized with each release of Blender. "The Core" will contain only those scripts of the highest quality and functionality; scripts that meet or surpass all of the standards created by the BPyC committee. Each script in The Core must be specially reviewed and approved by the committee (or a member of the committee at least).

2. The authors. Anyone can become an author in the BPyC. All that is necessary is that you create a login account. Additional restrictions may be imposed by the committee (such as the requirement that one submit an approved script within 30 days of their account's creation). The authors are responsible for updating and maintaining the scripts on the BPyC database. The BPyC committee will not ensure that the scripts on the database are the most up-to-date version available.

3. The users. The users are the Blender community at large. From their perspective, the BPyC will be a great big treasure chest of scripts. Mechanisms will be supplied to enable them to download individual scripts, download entire script packages (such as The Core), and update previously downloaded scripts.

-----------------------------------------------

And this is the conclusion of my initial proposal. Although it is fairly "high-level" in detail, I do have some lower, implementation-level details that can be presented later.
Initially, I wanted to get some feedback on the feasibility of such a project.

I will need the following feedback and support:
Committee - Although I am willing to initially be a member of the founding committee, my skills in Blender-Python programming are limited (although I do make an effort), and I will need support from several "scripting gurus." The more people we can add to the committee, the easier the load will be on each individual. Do I have support from any of you "gurus?"

Blender Foundation - I would love to see this integrated as a part of the Blender Foundation, but have not attempted to make any roads in there yet. We can always make the BPyC a separate organization than the Foundation, but I think that the closer we can get to Blender, to more accessible the BPyC will be. What are the possibilities of having this integrated with the Foundation?

Authors - Without dedicated authors, the main purpose of the BPyC (a central script database) will never be realized. It will be your responsibility to maintain your scripts on the BPyC server. Are you willing to do that, or would you rather continue to work independently?

Web site - I had initially been planning to get this up and running myself. However, I am a newbie to web programming, and have had almost no experience with maintaining a full-blown web site or renting a server. I've never even personally rented a domain name before. Although I am willing to start the work and have done some initial design, I will need support from web programmers and developers. There may be some advantages in this aspect if the BPyC is integrated with the Foundation and made a part of the Foundation's website. Will I have support from web programmers/developers? Will we be able to acquire the funds necessary to host the database?

Advertisement - This venture can only succeed if it is publicized properly. How can we ensure that every author and potential script user be made aware of the BPyC's presence?

Download manager - Basically, my ideas for a "download manager" stem from such systems as the cygwin setup program or the Firefox extension manager. A system that will enable users to view installed scripts, view and download new scripts, and update previously installed scripts. I believe that SamAdam has made a few strides in this area, but I could use feedback from other programmers as well.


Feedback, feedback, feedback. I need to hear from you. I am a busy person (aren't we all), and although my vision for this project is strong, without support I will have no choice other than to pass on the legacy, or let it die.

Thanks!
Levi
A three-legged stool never wobbles.

levon
Posts: 0
Joined: Thu Jul 31, 2003 6:06 am
Location: adelaide

Post by levon »

great idea, its something that has been talked about before, shortly before the 2.37 release it was decided that there will be a new 'release' of python scripts, so that the ones included with blender could be mainly importers and exporters, and the other download would be all sort of python scripts. the other benefit is this can be released at different times of blender, so scripts can be updated more often. although i didnt read your entire post, i think a better system of submit scripts would be nice, and also a way of knowing that the scripts are up to date etc.... if you have ever used miranda IM, it has a system where plugins are all in a centralised server, and there is a plugin that will look on the server to make sure you have the most update scripts, and if not it tells you and can install them for you. iirc someone made a python script for blender that would do this.

ill have another read of your proposal soon... and you also might want to join, and then post it to the bf python mailing list http://projects.blender.org/mailman/listinfo/bf-python

reD_Fox
Posts: 0
Joined: Sat Mar 12, 2005 2:56 pm
Location: Abilene, KS

Post by reD_Fox »

levon wrote:although i didnt read your entire post
Sorry it was so long. In reflecting back on it, it wasn't even very organized. I probably would have done better posting a summary.
levon wrote:if you have ever used miranda IM, it has a system where plugins are all in a centralised server, and there is a plugin that will look on the server to make sure you have the most update scripts, and if not it tells you and can install them for you.
I have never used or even heard of "miranda IM," but it sounds like it has a system almost identical to what I'm proposing. Cygwin uses something similar to download and manage its programs, and Firefox uses a similar system as well for its extension manager.

However, I would like the system to be more than just a "script database." I would like to see the promotion of coding standards, the use of an author-maintained database, and the encouragement of "scripting teams" (the MakeHuman project, if nothing else, should show us the power of a multi-person scripting effort) and other types of colaboration.

It's encouraging to hear that the Foundation is already thinking about moving in this direction. I'll see about refining the proposal and posting it to the mailing list. Although I would love to see this integrated as a part of the Foundation, I have not been active in Blender's development, and don't really know how the Foundation's "inner workings" are accomplished. Hence, my proposal (which was built from scratch) may need to be adapted for the Foundation's environment.

Levi
A three-legged stool never wobbles.

joeri
Posts: 96
Joined: Fri Jan 10, 2003 6:41 pm
Contact:

Post by joeri »

I think your plan is alread possible within the foundation.
If you are willing to use the wiki.

If not then we will have to discuss the savety etc of uploading scripts etc.
And a major part is keeping the BPyC area alive.
I'm willing to help out as a designer and/or php programmer (after my holiday).

btw, Is www.BPyC.com still free?

Brandano
Posts: 0
Joined: Mon Apr 19, 2004 6:03 pm

Post by Brandano »

joeri wrote:btw, Is www.BPyC.com still free?
Nope, I am afraid but the BURNHAM PARK YACHT CLUB might not like the idea of someone trying to buy off their domain :)

suntzu
Posts: 0
Joined: Wed Feb 18, 2004 6:20 pm
Location: Berlin
Contact:

Post by suntzu »

But www.BFPyC.com is free.

suntzu

Nostalghia2
Posts: 0
Joined: Tue Jul 12, 2005 11:14 pm

how to find my users

Post by Nostalghia2 »

Hi,

I am a script author. Some month ago I had to decide where to host my script and the tutorials. First I thought it where better to host it in the project tree at http://projects.blender.org/ because of the community. But after some mails with Ton I decided to host it at http://sourceforge.net. It's my second open source project at sourceforge.net. Their service is good and free. And I can concentrate on development and they took care running the service.

But the question how to find my users is still open. Here BPyC can be a great help. Not as an new and seldom visited location in cyberspace, but as part of Blender.org.

A download and update manager in Blender makes sense. One requirement is that the manager could handle the situation that my script is splited in several files.

Some month ago I read an article from an angry man explaining that there are 43 text editors projects at sourceforge.net. He said it's waste of time. In deed for open source projects I am using only two different editors: Eclipse and JEdit. But on the other side there is more than one of terrain generation scripts based on blenders noise functions. And I am one of the author making his own terrain generator (in alpha state, so sorry). What can I say? BPyC can be helpful to get a general survey. But it could not circumvent that there are many simple scripts covering the same functionality.

What makes sense for me - in short words:
1 A catalogue with available scripts
1.1 A ordering schema with categories equivalent to the “Scripts” button in Blenders script window.
1.1 A short description for every script.
1.2 A screen shot.
1.3 A link to a tutorial or home page.
1.4 A hint from the script developer about the state of his script. "idea”, “alfa”, “stable”...

2 An update manager inside Blender.
2.1 Able of handling multi file scripts.
2.2 Removing outdated *.pyc files
2.3 One directory for every script plugin to avoid name clashes.

Best Regards
Thomas Jourdan
http://kandid.org

reed
Posts: 0
Joined: Thu Jun 17, 2004 4:07 pm

Post by reed »

I like the motivation here, we need a way to keep track of scripts. But I think the most important thing is to have a script repository at www.blender.org. The repository must be a new part of blender.org-- if it's a seperate effort, the repository will vanish if the volunteers behind that effort lose interest or otherwise move on. All scripts must be hosted on the repository (no links to random other websites). If the repository is part of blender.org, it exists as long as Blender does, and new volunteers can come in to replace old ones.

The new menu-metadata at the top of new scripts will help a lot in making an automated web page. There could be a place on the repository you can "submit" a script for inclusion in the repository, where it's uploaded and stored in a "pending" section until a volunteer checks it and moves it into the main section with the rigtht author, copyright, category, version number, blender version, date, etc.

Many scripts have dedicated projects at projects.blender.org, but it's not really the purpose of projects.blender.org to have a project dedicated to each and every little tiny script. Maybe a "special" project could be created to host the repository? Or a new section of the blender3d.com CMS might be better?

Anyway, great idea, it would do so much for Blender, scripts are becoming a crucial part of it. I hope this script collection can become a permanent part of the Blender website.


reed

reD_Fox
Posts: 0
Joined: Sat Mar 12, 2005 2:56 pm
Location: Abilene, KS

Post by reD_Fox »

He, he. Yeah, I was originally thinking www.BPyC.org , but what do you know, it's owned by a yacht club, too.
joeri wrote:I think your plan is alread possible within the foundation.
If you are willing to use the wiki.

If not then we will have to discuss the savety etc of uploading scripts etc.
And a major part is keeping the BPyC area alive.
I'm willing to help out as a designer and/or php programmer (after my holiday).
I didn't think the wiki supported a dynamic system (i.e. a PHP driven web interface). So aside from posting preliminary project stats, I hadn't considered it.
Umm, "savety" (new word?)? Sorry, I don't get it.
You're right about keeping the BPyC alive. It's going to take a lot of effort to get started, and I wanted to make sure it would have the needed support from authors and users (especially authors) to become a viable option.
The way I look at it, there's no big hurry. I'm busy, and I think most everyone else is too. We've gotten along this far without the BPyC, and I can imagine we'll continue to get along pretty well without it (although I imagine we'll get along better with it). Have fun on your holiday!
Nostalghia2 wrote: A download and update manager in Blender makes sense. One requirement is that the manager could handle the situation that my script is splited in several files.

Some month ago I read an article from an angry man explaining that there are 43 text editors projects at sourceforge.net. He said it's waste of time. In deed for open source projects I am using only two different editors: Eclipse and JEdit. But on the other side there is more than one of terrain generation scripts based on blenders noise functions. And I am one of the author making his own terrain generator (in alpha state, so sorry). What can I say? BPyC can be helpful to get a general survey. But it could not circumvent that there are many simple scripts covering the same functionality.
I think the BPyC could easily meet the need for multiple script files. I've even considered the possibility of, in the future, giving authors the option to sell their scripts (careful...) via licenses and such.

The point you make regarding duplication of effort is a good one. The BPyC cannot (and should not) limit the duplication of effort. However, in your case, I would rather see you team up with another landscape script author, combine your ideas, and come up with a landscape generator that is twice as good, rather than continue to work by yourselves and produce two lesser-quality scripts. Then again, competition and the user's ability to choose between different "flavors" of the same type of product can be a positive thing.
My point is, even though colaboration should not be mandated, the BPyC should set up an environment where it is encouraged.

Your list of "requirements" is a good one, and I have put it on file. In many points you parallelled points that I had written down previously, but you also made some excellent suggestions that I hadn't thought of.

Speaking to you as an author, if the BPyC were to become a reality, would you be willing to give the extra effort to maintain your scripts in two different locations, as well as provide and update the needed meta-data (e.g. points 1.1 - 1.4)?
Now, would you still be willing to do it if you had a large number of scripts?
Take jms's script repository, for instance:
http://www.zoo-logique.org/3D.Blender/i ... p3?zoo=pyt
The answers of script authors to questions such as those will, I believe, determine the success or demise of the BPyC.

reed, I agree with everything you've said (and posted a reply to your post on elysiun).

I've been researching some web hosts/servers. Does anyone have any thoughts on what the storage/bandwidth requirements would be for a project such as this?
I'm needing something where I can at least build a prototype/sand-box for testing. For starters, I thought that the "prologic" plan at http://www.startlogic.com/ would be good. Any comments, warnings, or suggestions?

Thanks!
Levi
A three-legged stool never wobbles.

paleajed
Posts: 0
Joined: Fri Feb 11, 2005 1:19 pm

Post by paleajed »

Great initiative !

I remember how long it took me to get to know where to get all the good scripts. :?

Most people just use the included scripts, some get a little bit more curious and look at the scripts presented at www.blender3d.com, some even dare venturing to the forums where the really tasty fruits can be tasted.
But the forums are no centralized effort and interest for new scripts quickly dwindles when the topic announcing the script drops off the main message page. I am a script writer myself (don't call me a guru or I'll call you a baba) and think it to be extremely funny to see how interest for the script goes up every time I post a message on the forums. Seems like only people that are really active on the forums (visit every week) know about and use the new script releases.

But on a more important note: some weeks ago LetterRip posted the Blender's Digest to main page blender.org news, and this caused interest in my scripts to go through the roof for a few weeks (again, I believe until the post got out of focus). This clearly shows accessibility is the key to showing people the way to scripts. People just don't want to (or don't have the time) to dig to deep, figuring everything out themselves.
Blender 2.37 got 100.000 downloads the first week! 90% of these people NEVER download a script and I believe most don't know about all these little treasures that could help them becoming more productive.

I read some ideas on the forum before about making a script database, but never as well presented as yours. Don't give up on this initiative. DO get the forum moderators to make your post STICKY on the Python forum, it needs to be, because of its importance for the whole Python scripting community (maybe the idea needs more credibility first ?).

In the spirit of this answer, I would propose to add a menu command "File->Import->Download New Script" to Blender itself (which would launch your standard browser to the script database page) to keep the treshold for the user as low as possible. If this is going to far, we'll need at least integration through the main Blender webpages.

And about helping out, I'd sure make full use of your system for my own scripts. Maybe I can do more to help, but I'm still deciding if I'll be turning my attention toward Blender again (forgot about it for three full months). Yeah, sometimes I just hate computers.


Whatever, keep at it, I'm off to the Rainbow Gathering.




my lil' scripts: http://users.telenet.be/EWOCprojects
________________________________
Don't feed your ego, feed the duckies...

Post Reply