Questions about the datastore location
February 17, 2010 – 8:26 amIt should not be this hard for people to figure out how to get large numbers of files from their journal to another computer. When I think “hm, where would the picture I took with Record be stored?” my first instinct is not “of course, it must be ~/.sugar/default/datastore/store with cryptic alphanumeric-hashed filenames and no file extensions!”
Would it be possible to…
- Move the default “Journal stores things here” folder to ~/sugar/MyJournal or something like that? (Note ’sugar’ instead of ‘.sugar’ – not using a hidden folder makes this more discoverable. I wonder if this is more than a global find-and-replace of the path for any reason.)
- Make filenames of Journal items more identifiable – Karlie suggested using the first-created date and time (YYYYMMDDHHMMSS) as the beginning of the hash, and appending the filename extension, so for instance 20100217082415-asf693df7.png would at least tell me that this is an image I took on Feb 17, 2010, at 8:24:15am.
I know these things are probably never going to be seen by the vast majority of kids using Sugar, because we’re trying to leave behind the file-and-folder metaphor, but Sugar is built on an OS that does use the file-and-folder metaphor, so this would make it easier for the hackers tinkering on the undersides to figure out the underlying structure for themselves.
Are there design decisions behind the current Journal storage format that I just don’t know about? I’m honestly curious – it may be that a nested series of hidden folders is the right way to go for some reason I haven’t figured out, and that it’s worth the tradeoff – but right now I can’t think of any.
4 Responses to “Questions about the datastore location”
Years ago on the XO, I encountered the same problem of doing bulk examination and export.
And ended up using a stack of one-liners something very vaguely like… oh, this:
http://wiki.laptop.org/go/Talk:Release_notes/623 , and a perhaps more official one:
http://wiki.laptop.org/go/Sugar.datastore.datastore#How_do_I_get_command_line_access_to_the_files_in_my_DataStore.3F .
I wonder how many times people have rolled their own solutions to this one. :( I too then moved the copies to a normal linux box to use its file browser.
Though I’m usually a big fan of self-documenting file names and non-binary protocols, here the underlying problem seems not the datastore filename, but the lack of an Activity to easily do bulk examination and movement of one’s work. Journal was back then unusably poor at this use case. And apparently still is. Perhaps a new “artifact-centric view” App is needed to ‘get it right’? And worry about tweaking Journal later, as it’s already so heavily committed, and thus has conflicts of interest? Though, perhaps the functionality is already on the Journal todo bug list?
Architecture wise, it’s nice to keep unique identifiers simple, clean, and without additional semantics. Eg, numeric uids in an sql database – one could start overloading them with other information (eg, “blue-happy-widget-123″ instead of “123″), but this would complexify without helping with much. So in the datastore, is the path of minimum complexity considering them more like uids, or more like archival files which should be self describing? And if archival, a common means of description would be adding a .meta file… which the datastore likely already has… which brings us back to tools.
By Mitchell on Feb 17, 2010
Some information about the layout:
http://wiki.sugarlabs.org/go/Development_Team/Datastore_Rewrite
What I think we should do, is to develop a nautilus/gio/fuse plugin that reads the metadata and exposes it in the filesystem.
Here is a FUSE plugin in C#:
http://git.sugarlabs.org/projects/fsgateway/repos/mainline
This would be a great addition to the OLPC builds with both GNOME and Sugar
By Tomeu Vizoso on Feb 17, 2010
The short version is “no, it doesn’t have to be this way!”.
The long version is, as always, “it’s complicated, though in this case, the way forward is comparatively well described.”
In short, the recipe calls for two parts Journal redesign, one part data store replacement, and one part network redesign.
Links to some details:
* http://wiki.sugarlabs.org/go/Design_Team/Designs/Journal
* http://wiki.laptop.org/go/Journal2
* http://wiki.laptop.org/go/Olpcfs
* http://wiki.laptop.org/go/Network2
Questions?
By Michael Stone on Feb 17, 2010
Some meta observations…
Current: Reoccurring need exists, adult programmer maybe checks wiki for hints, uses command-line, then maybe blogs or wikis how-to hints.
State of art: Reoccurring need exists, adult checks AppStore, downloads App. Apps are hard to create, share, or modify (as are sugar Activities).
Future: Reoccurring need exists, kid tells Helpful Tool Bench, questions are asked, need met, questions become easier.
Incremental improvement of current:
* “maybe checks” – strengthen “of course I checked X first!” social focus (eg, AppStore).
* “wiki” – lower barrier of difficult unstructured unaided search experience. “I want to …” task oriented search. Clarifying “do you want Y or Z?” aids.
* “hints” – ease path for hint becoming script becoming curated and available ($ olpc-contrib3 –bobs-flickr-uploader help;) becoming polished becoming sugared gui. AppStore. And ideally, not resulting in rpm/AppStore/Activity “here’s a pile of discrete tools, choose one and read its docs”, but a much more fine-grained task-oriented environment. Though, python doesn’t make it easy.
* “command-line” – supportive IDE. Success when “why would you want to go down to the CLI level when you can simply do…?!?”.
* others…
Minor matter of developer resources, and incentives, and clarity of vision, and less crippling programming languages, and… opportunities for incremental improvement.
By Mitchell Charity on Feb 19, 2010