TableOfContents

Lumiera/ConfigLoader is a brainstorming about our configuration file loader.

API

 lumiera_config_save () { LLIST_FOREACH(config_singleton.files, f) { LumieraFile file = (LumieraFile) f; if(lumiera_configfile_isdirty (file)) lumiera_configfile_save(file); } } 

Notes:

Example:

 integer_set("foo.bar", 123)   looks up "foo.bar" this returns as well file and line it will be decided whether it's a system file or a user's config file. if systemfile: loop for user's config file, if necessary create it in our RAM but not yet on disk set the value in the line.

lumiera_configfile

provided TYPES

config type

C type

semantic (example)

number

signed long long

also hex and octal (foo.bar = 12345 #comment)

real

long double

differerent formats as well (foo.bar=1.2345 #comment)

string

const char*

optionaly quoted string (foo.bar="this string" #this not)

word

const char*

first optionaly quoted word of the config (foo.bar= first this # not)

bool

int

common representations for bools (0,1,true,false,yes,no,on,off,set,clear)

... add more types on demand (ichthyo mentioned a tuple type for color triples etc.)

config files/strings are utf-8!

Syntax/Semantics

...to be defined...

in short:

Syntax Proposal by cehteh

simple example:

# comment until end of line
@directive options
key = value
[prefix]
[prefix suffix]

ebnf:

configline ::= [ whitespace ] , { comment | directive | configentry | section } , ? end of line ? ;

whitespace ::= ? one to many tab or blank ? ;

comment ::= "#" , ? arbitary text ? ;

directive ::= "@", ? commandword ? , whitespace, ? arguments ? ;

configentry ::= key, { whitespace } , assignop, value ;

key ::= ? lowercase characters and _ ? , [ "." , key ] ;

assignop ::=  " " | ":" | "=" ;

value ::= ? arbitary text ? ;

section ::= "[", [ whitespace ] , prefix, [ whitespace, suffix ] , [ whitespace ] "]" | emptysection ;

emptysection ::= "[]" ;
prefix ::= key ;
suffix ::= key ;

semantic notes:

http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html

internals/implementation sketch

discussion on irc

[01:21] <cehteh> joelholdsworth_: you seen the upcoming config interface i planned with simav?
[01:21] <mno> what about using ardours icons to start with?
[01:21] <joelholdsworth_> *least
[01:22] <joelholdsworth_> cehteh: not yeat, can I see
[01:22] <joelholdsworth_> mno: ardour icons arn't that useful
[01:22] <joelholdsworth_> there aren't that many
[01:22] <cehteh> http://www.pipapo.org/pipawiki/Lumiera/ConfigLoader
[01:22] <cehteh> question for you: do you need any more 'types'
[01:22] <joelholdsworth_> and those that are there are probably among the gtk stock set anywhere
[01:22] <cehteh> ichthyo suggested color triples
[01:23] <cehteh> anythinfg you have in mind, just suggest it
[01:23] <joelholdsworth_> so is the this structured like an ini file
[01:23] <joelholdsworth_> sections and keys?
[01:23] <joelholdsworth_> or is it treelike?
[01:24] <cehteh> little unordered, you basically just need to care for lumiera_config_TYPE_get ... and *_set
[01:24] <cehteh> like ini, not nested
[01:24] <joelholdsworth_> flat is good
[01:24] <joelholdsworth_> simple
[01:24] <cehteh> the namespaces are nested
[01:24] <cehteh> but the sectioning is not
[01:24] <cehteh> you can have
[01:24] <cehteh> [gui]
[01:24] <__nasa__> Have you all see Agave (for colorscheme generation)?
[01:24] <cehteh>  foo = bar
[01:25] <cehteh> [gui.baz]
[01:25] <cehteh>  barf= 123
[01:25] <cehteh> its not exactly ini style but close
[01:25] <joelholdsworth_> cehteh: ooh - does mean we can levarge an ini parser?
[01:25] <joelholdsworth_> that would save some work and maintenance effore
[01:25] <cehteh> hey simav want to write the parser himself :P
[01:25] <joelholdsworth_> __nasa__: agave: interesting
[01:26] <joelholdsworth_> cehteh: would he though
[01:26] <cehteh> well i think thats fun to do by ourself :)
[01:26] <joelholdsworth_> it is just like ini
[01:26] <joelholdsworth_> yeah
[01:26] <cehteh> and i plan some special things
[01:26] <joelholdsworth_> but reduction of volume is good
[01:26] <cehteh> like for example setting values from the gui preserves the rest of the data
[01:26] <cehteh> when you have
[01:26] <__nasa__> cehteh: suffixes?
[01:27] <cehteh> [gui.foo bar]
[01:27] <cehteh>  dada = 1234 #comment
[01:27] <__nasa__> I just saw the bottom...
[01:27] <cehteh> that is    gui.foo.dada.bar = 1234 # comment
[01:27] <cehteh> see suffix
[01:28] <cehteh> more in a moment
[01:28] <joelholdsworth_> still an ini parser would be happy with that
[01:28] <cehteh> what i want to tell to joel
[01:28] <joelholdsworth_> of course you have to add some extra code to the end to make the tree
[01:28] <joelholdsworth_> but that would make our lives simpler
[01:29] <cehteh> when you now have a gui which does gui_config_number_set ("gui.foo.dada.bar", 6666)
[01:29] <cehteh> then
[01:29] <cehteh> [gui.foo bar]
[01:29] <cehteh>  dada = 6666 #comment
[01:29] <cehteh> appears in the configfile
[01:29] <joelholdsworth_> did you mean...
[01:29] <cehteh> meaning preserving everything manually set up
[01:29] <joelholdsworth_> [gui.foo.dada]
[01:29] <__nasa__> cehteh: How does it know not to do
[01:29] <joelholdsworth_> bar = 6666 #comment
[01:29] <joelholdsworth_> ?
[01:30] <__nasa__> [gui dada.bar]
[01:30] <__nasa__> foo = 6666 #comment
[01:30] <__nasa__> joelholdsworth_: that works as well :)
[01:31] <cehteh> it just selects best fit sections which already exist
[01:31] <joelholdsworth_> but isn't it supposed to be this way...
[01:31] <cehteh> best fit .. first prefix, then suffix
[01:31] <joelholdsworth_> so [foo.bar] is different from [bar.foo]
[01:31] <__nasa__> Ok.
[01:31] <cehteh> if nothing fits then it places it at a non sectioned part in verbose way
[01:31] <cehteh> joelholdsworth_: yes
[01:31] <__nasa__> (I just wanted to make sure we had a decision on that, as I agree that is the best way)
[01:31] <cehteh> so rationale about sections:
[01:32] <cehteh> rather rarely used but you can do things like
[01:32] <cehteh> [plugin radius]
[01:32] <cehteh>  blur = 1
[01:32] <cehteh>  sharpen = 2
[01:33] <cehteh> or same for key shortcuts what simav initiall intended
[01:33] <joelholdsworth_> wouldn't it be simpler to have simpler systaxx
[01:34] <joelholdsworth_> and have it always with dots or with spaces
[01:34] <cehteh> well the prefix stuff is completely optinal
[01:34] <joelholdsworth_> that way you can't have 1 section in twice
[01:34] <cehteh> eh?
[01:34] <joelholdsworth_> well is [plugin radius] the same as [plugin.radius] ?
[01:34] <cehteh> no
[01:35] <joelholdsworth_> good
[01:35] <joelholdsworth_> ok
[01:35] <cehteh> [plugin radius]  would be suffix prefix
[01:35] <joelholdsworth_> seems overkill to me
[01:35] <cehteh> err opposite
[01:35] <joelholdsworth_> I'd just stick with one or the other
[01:35] <cehteh> <tired
[01:36] <cehteh> [plugin radius]  would be prefix suffix
[01:36] <cehteh> [plugin.radius]  is just prefix
[01:36] <mno> i'm trying to understand all this hehe
[01:37] <cehteh> the idea is that with just prefix you have to settle with a quite rigid hierachy where you know that things will always be ordered by the leaves
[01:37] <cehteh> that doesnt fit for everyone
[01:38] <joelholdsworth_> "ordered by the leaves" ?
[01:38] <cehteh> one might want to configure his plugins one affter each other
[01:38] <__nasa__> Here is what I think is cool about the setup: it means that we can treat builtins and plugins the same.
[01:38] <cehteh> imagine we have    blur and sharben
[01:38] <cehteh> sharpen
[01:38] <__nasa__> [plugin radius]
[01:39] <__nasa__> cinelerra.sharpen = 3
[01:39] <cehteh> both have a radius and a algorihtm. lets say horizontal or vertical
[01:39] <__nasa__> rotate = 2
[01:39] <cehteh> so then you can either first configure blur
[01:39] <cehteh> [plugin.blur]
[01:39] <cehteh>   radius = 1
[01:39] <cehteh>   algo = horiz
[01:40] <joelholdsworth_> oooh! - have you thought of simply using libconfig?
[01:40] <cehteh> # and then sharpen
[01:40] <cehteh> [plugin.sharpen]
[01:40] <cehteh>   radius = 2
[01:40] <cehteh>   algo = horiz
[01:40] <cehteh> ..
[01:40] <__nasa__> joelholdsworth_: that format looks a little painful, but it could work.
[01:41] <joelholdsworth_> it's like JSON I think
[01:41] <cehteh> hey so far thats just like ini
[01:41] <cehteh> well now about suffix
[01:41] <joelholdsworth_> ok
[01:41] <cehteh> imagine you want to configure things by 'topics' ..
[01:42] <cehteh> # first define radius for our plugins
[01:42] <cehteh> [plugin radius]
[01:42] <cehteh>   blur = 1
[01:42] <cehteh>   sharpen = 2
[01:42] <cehteh> # then algo
[01:43] <cehteh> [plugin algo]
[01:43] <joelholdsworth_> yes that's quite clever
[01:43] <cehteh>   sharpen = horiz
[01:43] <cehteh>   blur = horiz
[01:43] <joelholdsworth_> but still it seems overkill
[01:43] <__nasa__> joelholdsworth_: It does look similar to JSON, but more C-like. For example, there are semi-colons instead of commas at the ends of single values.
[01:43] <cehteh> well its completely optional and we can leave it out
[01:43] <joelholdsworth_> these files are not primarily for humans
[01:44] <cehteh> yeah
[01:44] <joelholdsworth_> and also that prefix thing while clever is quite non obvious when you first see it in the file
[01:44] <__nasa__> Yes, but I am a big proponent of trying to make things human readable
[01:44] <cehteh> git works the same way
[01:44] <joelholdsworth_> sure it should be human readable
[01:44] <cehteh> .git/config
[01:44] <__nasa__> So if anything does go wrong, the user can fix it.
[01:45] <joelholdsworth_> well I'm not sure the prefixes thing is obvious enough
[01:45] <cehteh> well and we will have plenty of values which dont need a gui
[01:45] <cehteh> see about:config for firefox
[01:45] <__nasa__> I think that INI is much easier to read than libconfig, even though I have been programming in C for years and I have never edited a windows INI file.
[01:46] <joelholdsworth_> I think I agree
[01:46] <cehteh> they have all defaults and only when altered they are recorded to a config file but they dont need any special gui
[01:46] <__nasa__> about:config is so nasty.
[01:46] <cehteh> well its mighty
[01:46] <joelholdsworth_> but yes all of about:config could be stored in ini format
[01:46] <joelholdsworth_> [accessibility.typeaheadfind]
[01:46] <cehteh> and anything which is commonly changed should have a preferences gui
[01:46] <joelholdsworth_> linksonly=false
[01:47] <cehteh> btw our configs are not enumerateable
[01:47] <cehteh> i dont want the rest of the system to register there, that will be pita
[01:47] <cehteh> you can freely introduce configs (within your namespace)
[01:48] <-- benG has quit (Remote closed the connection)
[01:48] <cehteh> i'd rather make a grep script which goes through the sources and searched for   lumiera_config_.*_get (".*", .*)
[01:49] <joelholdsworth_> yeah that's ok
[01:49] <cehteh> to have some idea what configs are actually used
[01:49] <joelholdsworth_> well maybe we can use libini
[01:49] <cehteh> so a about:config like ui wont be possible
[01:49] <__nasa__> which libini? There are quite a few.
[01:49] <joelholdsworth_> ahh
[01:49] <joelholdsworth_> that's a shame
[01:49] <cehteh> he forget about some libs .. thats so trivial to write by ourself :P
[01:49] <__nasa__> I assume the SF one is the best, as it is GPL.
[01:50] <cehteh> and simav wants to do something :P
[01:50] <joelholdsworth_> yeah
[01:50] <joelholdsworth_> anyway... time for bed for me
[01:50] <cehteh> well and what i rate high is that one can retrieve typed values  and that one can add/set values
[01:51] <cehteh> and this setting should not destroy the existing file structure
[01:51] <joelholdsworth_> yeah
[01:51] <cehteh> just read and write values is no magic .. but preserving human structured files is
[01:52] <joelholdsworth_> that is hard, yes
[01:52] <joelholdsworth_> anyway... got to go
[01:52] <__nasa__> Bye!
[01:52] <cehteh> n8 :)
[01:52] <-- joelholdsworth_ has quit ("Ex-Chat")
[01:52] <__nasa__> It looks like libini uses [plugin/blur] instead, but that is an easy fix.
[01:52] <cehteh> url?
[01:52] <__nasa__> Or we just let simav do the whole thing
[01:53] <cehteh> well i dont know how much time he has
[01:53] <__nasa__> http://sourceforge.net/project/showfiles.php?group_id=25464
[01:54] <cehteh> last change 3 year ago :P
[01:56] * cehteh looks at the source
[01:57] <cehteh> fails standard C++ conformance :P
[01:58] <cehteh> and no @include like directive
[01:58] * __nasa__ thinks we should look elsewhere...
[01:58] * cehteh things we should just do ourself :P
[01:58] * __nasa__ agrees.
[01:59] <cehteh> the syntax i typed there is not very special and can be replaced
[01:59] <cehteh> but some other features are important
[01:59] * __nasa__ thinks it should PRE. That way we don't have to totally agree.
[01:59] <cehteh> pre?
[01:59] <__nasa__> (On syntax)
[01:59] <cehteh> no suffix?
[02:00] <cehteh> well as saied its just an idea and completely optional, we can leave it out
[02:00] <__nasa__> sorry -- PCRE.
[02:00] <cehteh> ah
[02:00] <cehteh> i dont see how PCRE helps for a syntax
[02:00] <__nasa__> I always think of it as "Perl Regular Expressions" and forget the "Compatible"
[02:01] <cehteh> anyways: features matter
[02:01] <__nasa__> I was thinking in terms of plugins/blend vs. plugins.blend.
[02:01] <cehteh> that is @include which follows some LUMIERA_CONFIG_PATH
[02:02] <__nasa__> We should have some way to only include certain sections.
[02:02] <__nasa__> Like have the default config be
[02:02] <cehteh> yes
[02:02] <__nasa__> [lumiera.core]
[02:02] <__nasa__> ...
[02:02] <__nasa__> then
[02:02] <cehteh> but thats setup by the administrator/user
[02:02] <__nasa__> @include<plugin> ~/.lumi/plugins/
[02:02] <cehteh> if the user changes lumiera.core.somevalue
[02:03] <cehteh> it will find the file where it is already defined
[02:03] <__nasa__> and it adds all preferences for plugins from <dir>
[02:03] <cehteh> then check if thats a file in the users config space (~/.lumiera/*)
[02:03] <cehteh> and edit it there
[02:03] <__nasa__> Ok.
[02:04] <cehteh> if it is a system wide file it will place a correspondending file in the users config space
[02:04] <cehteh> if the value is not configured already it will search for [lumiera.core] and then [lumiera] and then []
[02:04] <cehteh> best fit the section where it would apply
[02:05] <cehteh> and then do the same write or create into correspondending user config
[02:05] <cehteh> thats why i saied 'features matter'
[02:06] <__nasa__> There should be a callback for a plugin to add a config file for that plugin.
[02:06] <__nasa__> Ooh
[02:06] <__nasa__> cool idea
[02:06] <cehteh> this is not really complicated to do but would give a big usage experience JustWorks[TM]
[02:06] <__nasa__> ~/.lumi/config is the main configuration directory
[02:06] <__nasa__> inside, there is a ~/.lumi/config/config.ini which is main configuration
[02:07] <cehteh> no callbacks .. the idea is that the config system is independent and nothing else need to register there, that makes it much easier
[02:07] <__nasa__> Then, for example accessing plugin.shade.
[02:07] <cehteh> if you install a plugin this plugin should come with a system config file
[02:07] <cehteh> that can be a empty one
[02:07] <__nasa__> either find it in ~/.lumi/config/config.ini or ~/.lumi/config/plugin/config.ini.
[02:08] <cehteh>  /usr/local/share/lumiera/plugin/shade
[02:08] <cehteh> containing just
[02:08] <cehteh> [plugin.shade]
[02:08] <cehteh> when the user edits some value
[02:09] <cehteh> plugin.shade.transparency = 209
[02:09] <cehteh> then this will automatically create a correspondending ~/.lumiera/plugin/shade  in the user config space
[02:09] <__nasa__> yeah, so what I am saying is that instead of [plugin.shade] only [] is needed
[02:10] <cehteh> using the system one as template
[02:10] <cehteh> well if its []
[02:10] <__nasa__> since the file is already in config/PLUGIN/SHADE
[02:10] <cehteh> or that
[02:10] <cehteh> this is not yet decided
[02:10] <cehteh> we can do that too
[02:10] <__nasa__> so automatically add [plugin.shade] to the prefix of files in config/plugin/shade, etc
[02:11] <cehteh> but i would be careful to tie filenames and internal namespaces together
[02:11] <cehteh> yes doable
[02:11] <cehteh> well should still have [plugin.shade]
[02:12] <cehteh> else we have a nested section mess and files loose their identity
[02:12] <__nasa__> Yeah, I just realized it would be a pain to have default configs
[02:12] <__nasa__> Since where does the file go? you have to look inside to see if it had [plugin.shade], tec
[02:12] <__nasa__> etc*
[02:12] <cehteh> anyways nothing is etched in stone yet, we are just brainstorming
[02:13] <cehteh> most important thing is the interface the applciation can use
[02:13] <cehteh> then we can make a mockup for that which only returns defaults
[02:13] <cehteh> then we can use it already even when the config loader isnt finished
[02:14] <__nasa__> We could have a warning if a file in e.g. config/plugin/blur had a section that was [plugin.shade] or something
[02:14] <cehteh> actually the inital plan was to bootstrap the config system by one single site config file
[02:14] <__nasa__> to prevent hackery
[02:14] <cehteh> but plugins are a good point .. they might have their own configs
[02:15] <cehteh> which is not @included by the site config
[02:15] <__nasa__> lumiera_config_t *plugin_shade.request_default_config() or something
[02:15] <cehteh> but the plugin itself loads it when the plugin is used
[02:15] <__nasa__> Yeah
[02:16] <cehteh> nah
[02:16] <cehteh> lumiera_config is just singleton
[02:16] <cehteh> after you loaded plugin.shade it becomes visible there
[02:16] <cehteh> thats way easier
[02:16] <__nasa__> that is lumiera_config_struct
[02:16] <__nasa__> ok
[02:16] <cehteh> not even the struct
[02:17] <cehteh> thats an implementation detail
[02:17] <__nasa__> yeah -- to deal with later
[02:17] <cehteh> the programmers use only   lumiera_coonfig_TYPE_get   or _set
[02:17] <cehteh> where TYPE is to be defined .. see my notes
[02:18] <cehteh> number real string word ... and whatever else we need
[02:19] <cehteh> so you get typed data back .. when you provide a default then you can be even sure that it is always valid
[02:19] <__nasa__> Yeah, I do like that structure.
[02:19] <__nasa__> It also makes mass configuration setup very easy.
[02:21] <cehteh> btw file must have identity ...
[02:21] <__nasa__> meaning?
[02:21] <__nasa__> like they are not all config.ini?
[02:22] <cehteh> i mean a config file has to define for what kinds of values it applies
[02:22] <cehteh> [plugin.shade]    for the plugin/shade file
[02:22] <__nasa__> Ok, I see.
[02:22] <cehteh> then changing a values there will create the correspondending user file
[02:23] <cehteh> *thinking*
[02:23] <cehteh> the user could write  [plugin.shade]
[02:23] <cehteh> into some other config file which then becomes target for that
[02:23] <cehteh> (best fit)
[02:24] <cehteh> just wondering if the user just writes [pluin]
[02:24] <cehteh> [plugin]
[02:24] <cehteh> well i am sure we solve that
[02:25] <__nasa__> [plugin]
[02:26] <__nasa__> shade.radius = 3
[02:26] <__nasa__> in like the plugin/blur folder?
[02:28] <cehteh> no in the users config somewher
[02:28] <__nasa__> oh
[02:28] <cehteh> ~/.lumiera/myconfig
[02:29] <cehteh> shall it pick and mirror /usr/share/lumiera/plugin/blur to the user config space
[02:29] <cehteh> or write it to the [plugin] section in myconfig
[02:30] <cehteh> well maybe we add a   [plugin*]    syntax which hints the best fit
[02:30] <__nasa__> I would think it makes more sense to add a plugin/blur folder
[02:30] <__nasa__> or else lumiera/myconfig would be huge after configuring one project ;-)
[02:30] <cehteh> yes .. thats what you think .. i just wondering how to give the user the control to fit it to his needs
[02:31] <cehteh> ~/.lumiera/all_my_pluginmyconfig
[02:31] <cehteh> [plugin*]
[02:31] <cehteh> hints the best fit algo
[02:32] <cehteh> well just thinking about optional things
[02:32] <__nasa__> I think I am confused -- how is [plugin*] different from [plugin]
[02:33] <cehteh> when you change plugin.blur.radius   then the best fit algo will search the section which covers it best
[02:33] <cehteh> that would be [plugin.blur] in  /usr/share/lumiera/plugin/blur
[02:33] <__nasa__> Oh, unless you have [plugin*]
[02:34] <cehteh> so it will create a correspondending ~/.lumiera/plugin/blur  file for you
[02:34] <cehteh> yes exactly
[02:34] <__nasa__> Ok, that makes sense. Good idea!
[02:34] <cehteh> [plugin*] will take precedence
[02:34] <cehteh> when you have that then you can hint it not to create many tiny files
[02:35] <__nasa__> so that way the config is in one file
[02:35] <__nasa__> yeah
[02:35] <cehteh> hehe the suffux thing comes in mind here
[02:35] <cehteh> [plugin* radius]     in ~/.lumiera/plugin_radius_conf
[02:35] <__nasa__> I was thinking suffixed are only allowed at the e.g. plugin/ level if they effect multiple plugins.
[02:35] <cehteh> well or as trivial as
[02:36] <__nasa__> Ok.
[02:36] <cehteh> [plugin* enable]     in ~/.lumiera/plugins_enabled
[02:36] <__nasa__> Fun!
[02:36] <cehteh>  blur = on
[02:36] <cehteh>  sharpen = off
[02:36] <cehteh> ah .. we need a bool TYPE
[02:36] <cehteh> *blink*
[02:37] <__nasa__> on/off or true/false?
[02:37] <__nasa__> Well, either way.
[02:38] <cehteh> bool
[02:38] <cehteh>
[02:38] <cehteh> int
[02:38] <cehteh>
[02:38] <cehteh> common representations for bools (0,1,yes,no,on,off,set,clear)
[02:38] <cehteh> we can add more
[02:38] <__nasa__> It should be reasonably good at guessing
[02:40] <cehteh> yeah .. i would like to make this little intelligent
[02:40] <cehteh> for example when you want to set something it might look at the existing thing before doing so and then flavor the set with that
[02:41] <cehteh> when a string was quoted, the new one will be quoted to
[02:41] <__nasa__> "Weaken the blur effect"
[02:41] <cehteh> if there was a bool with 'yes' then it will choose 'no' ..
[02:41] <cehteh> and so on
[02:42] <cehteh> this are generally just a few extra lines of code, nothing dramatic .. but that makes it much nicer in detail
[02:43] <__nasa__> Yeah, I think that it will improve the user experience
[02:43] <cehteh> ok i add this irclog to the wiki
[02:43] <__nasa__> for most people, editing a config file is scary
[02:43] <__nasa__> so making it as easy and friendly as possible is good
[02:44] <__nasa__> All right

Brainstorming

Lumiera/ConfigLoader (last edited 2009-03-07 22:28:00 by 80)