openZIM Developers Meeting at Wikimania 2011

[Manuel Schneider]

Dear all,

Wikimania is approaching and by that also our first developers meeting
in 2011!

The openZIM team is happy to invite you to the first (really)
multinational developers meeting. After three meetings in the center of
Europe with mostly people from that area participating we are now going
to meet at Wikimania.

Prior to Wikimania are special conferences. The openZIM meeting is on
August 2nd and 3rd - the two days right before Wikimania starts - at
Beit Hecht, part of the Wikimania venue.

Please sign up here and participate in the planning:

* http://wikimania2011.wikimedia.org/wiki/OpenZIM_Developers_Meeting

For dedicated offline people there is still budget left so we can help
you funding your participation at this meeting! Contact me for this.

I'd be happy to see you there!


Manuel

[Manuel Schneider]

Hi all,

here is the Etherpad documentation of the openZIM meeting at Wikimania
right now:

http://etherpad.wikimedia.org/openZIM-Developer-Meeting-2011


/Manuel

[Tomasz Finc]

Thanks for publishing this Manuel. Having this available while be
remote is huge.

--tomasz



On Tue, Aug 2, 2011 at 11:56 PM, Manuel Schneider

Post by Manuel Schneider
Hi all,
here is the Etherpad documentation of the openZIM meeting at Wikimania
http://etherpad.wikimedia.org/openZIM-Developer-Meeting-2011
/Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

[Tomasz Finc]

I'm going to pull out some of the discussion points from the notes for
wider discussion.

* there is little support from the WMF

: This is always going to be a subjective answer but lets keep in mind
that the WMF has been sponsoring work for improving Kiwix, expanding
the Collections extension to support openZIM, and improving Article
selection.

Each one of those plays a key role in the offline content pipeline.
Non engineering wise we've been connecting with the English, Spanish,
and Portuguese projects so that we can help release new collection.
We've also been working with eReadia and a whole host of others to
better understand the offline space. Everyone can always do more but
its important to choose what you work on and make that work well.

I think the bigger discussion point can be is if the WMF doing enough
and if not knowing why.

* the promised things like regular ZIM dumps sohuld be finally done by
WMF to support the spread of the software / project

: The current XML dumps is system is fairly broken and has been a
nightmare to manage over the last couple of years. And if you think
the XML system is bad then don't even look at the HTML generation
system. Instead of pushing more work into it we've opted to go for a
much simpler route. Allow the collections infrastructure to generate
full project files. Using the existing system we make the workflow
much easier for anyone wanting openZIm files while at the same time
not making our infrastructure more complicated then it needs to be.

* openZIM and Kiwix are hardly able to cope with the expectations due
to missing developers - not only software-wise but also concerning
documentation

Totally agree. We've made some amazing progress on both the Kiwix
betas and Collections extension but we can still do more. I think we
have to look at the engagement that we do (LinuxTag, Wikimania, Berlin
Dev Conf, beta testing, blog posts, etc) and really assess whats
working and whats not. The chapters can do a lot here to fund
initiatives and help run them.

* reimbursement for the server to build ZIM files

+1

* house the Kiwix server in a NOC

+1 .. I've said for a while that we should just move it over to
download.wikipedia.org

* restructure the Offline pages on Meta wiki:
http://meta.wikimedia.org/wiki/Offline_Projects

Who is owning this piece? I'd love to add my ideas about how to go about it.

* writing proposal for a contractor on openZIM

Looking over http://openzim.org/Roadmap its bit hard to know what's
happening post Wikimania. Whats the future look like?

Eager to hear other peoples thoughts.

--tomasz

Post by Tomasz Finc
Thanks for publishing this Manuel. Having this available while be
remote is huge.
--tomasz
On Tue, Aug 2, 2011 at 11:56 PM, Manuel Schneider

Post by Manuel Schneider
Hi all,
here is the Etherpad documentation of the openZIM meeting at Wikimania
http://etherpad.wikimedia.org/openZIM-Developer-Meeting-2011
/Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

[Emmanuel Engelhart]

Post by Tomasz Finc
* the promised things like regular ZIM dumps sohuld be finally done by
WMF to support the spread of the software / project
: The current XML dumps is system is fairly broken and has been a
nightmare to manage over the last couple of years. And if you think
the XML system is bad then don't even look at the HTML generation
system. Instead of pushing more work into it we've opted to go for a
much simpler route. Allow the collections infrastructure to generate
full project files. Using the existing system we make the workflow
much easier for anyone wanting openZIm files while at the same time
not making our infrastructure more complicated then it needs to be.

I agree, this is maybe the simplest way today... but the time we may
save now will have to be invest/waste in the future.
So, after many discussions, also in Wikimania, I'm really not convinced
by this approach and I think this is not the best way.

The way which is really efficient and sustainable is to work on DumpHTML:
* This is the faster solution
* This is trivial to deploy for Mediawiki admins (should also be the
case at the WMF)
* This is not too expensive to develop
* This allows to make sophisticated post-treatment if necessary
* You may modify the rendering in details (we work directly with the
mediawiki render engine)
* This is a simple and maintainable (works with hooks and skins) solution

That's the reason why I will propose a grant request to a chapter to get
that done for the summer 2012. Preparation of the grant request is
already started, feel free to make comment on the talk page:
http://www.kiwix.org/index.php/Mediawiki_DumpHTML_extension_improvement#Workpackage3:_Integrating_Collection_and_DumpHTML_extensions_and_new_features

Post by Tomasz Finc
* openZIM and Kiwix are hardly able to cope with the expectations due
to missing developers - not only software-wise but also concerning
documentation
Totally agree. We've made some amazing progress on both the Kiwix
betas and Collections extension but we can still do more. I think we
have to look at the engagement that we do (LinuxTag, Wikimania, Berlin
Dev Conf, beta testing, blog posts, etc) and really assess whats
working and whats not. The chapters can do a lot here to fund
initiatives and help run them.

I think, what is urgent is to find a solution with the WMF for openZIM
dev. ressources.
For Kiwix, I think what is essential is to get the chapters involved.
So, if the WMF continues to support Kiwix dev. in the manner they do
today, this is perfect.

Post by Tomasz Finc
* reimbursement for the server to build ZIM files
+1

Ok, what would be the best way to request that. At Kiwix we have two
servers;
* one rented for the online presence
* one in my guestroom to build the ZIM files. This is a server with a
disk array.

We speak her of the second one (I already get money for the first one).
It costs around 5000 euros.

Post by Tomasz Finc
* house the Kiwix server in a NOC
+1 .. I've said for a while that we should just move it over to
download.wikipedia.org

What is the NOC? I think we have to keep it in Europe and at the best in
the neighborhood of Switzerland.

Post by Tomasz Finc
* writing proposal for a contractor on openZIM
Looking overhttp://openzim.org/Roadmap its bit hard to know what's
happening post Wikimania. Whats the future look like?

I could prepare something if needed?

Regards
Emmanuel

[Rotem Simha]

Post by Tomasz Finc
I'm going to pull out some of the discussion points from the notes for
wider discussion.
* there is little support from the WMF
: This is always going to be a subjective answer but lets keep in mind
that the WMF has been sponsoring work for improving Kiwix, expanding
the Collections extension to support openZIM, and improving Article
selection.
Each one of those plays a key role in the offline content pipeline.
Non engineering wise we've been connecting with the English, Spanish,
and Portuguese projects so that we can help release new collection.
We've also been working with eReadia and a whole host of others to
better understand the offline space. Everyone can always do more but
its important to choose what you work on and make that work well.
I think the bigger discussion point can be is if the WMF doing enough
and if not knowing why.
* the promised things like regular ZIM dumps sohuld be finally done by
WMF to support the spread of the software / project
: The current XML dumps is system is fairly broken and has been a
nightmare to manage over the last couple of years. And if you think
the XML system is bad then don't even look at the HTML generation
system. Instead of pushing more work into it we've opted to go for a
much simpler route. Allow the collections infrastructure to generate
full project files. Using the existing system we make the workflow
much easier for anyone wanting openZIm files while at the same time
not making our infrastructure more complicated then it needs to be.
* openZIM and Kiwix are hardly able to cope with the expectations due
to missing developers - not only software-wise but also concerning
documentation
Totally agree. We've made some amazing progress on both the Kiwix
betas and Collections extension but we can still do more. I think we
have to look at the engagement that we do (LinuxTag, Wikimania, Berlin
Dev Conf, beta testing, blog posts, etc) and really assess whats
working and whats not. The chapters can do a lot here to fund
initiatives and help run them.
* reimbursement for the server to build ZIM files
+1
* house the Kiwix server in a NOC
+1 .. I've said for a while that we should just move it over to
download.wikipedia.org
http://meta.wikimedia.org/wiki/Offline_Projects
Who is owning this piece? I'd love to add my ideas about how to go about it.

Jessie Wild (cc) & myself had some ideas written in her notebook.

Post by Tomasz Finc
* writing proposal for a contractor on openZIM
Looking over http://openzim.org/Roadmap its bit hard to know what's
happening post Wikimania. Whats the future look like?
Eager to hear other peoples thoughts.
--tomasz

Post by Tomasz Finc
Thanks for publishing this Manuel. Having this available while be
remote is huge.
--tomasz
On Tue, Aug 2, 2011 at 11:56 PM, Manuel Schneider

Post by Manuel Schneider
Hi all,
here is the Etherpad documentation of the openZIM meeting at Wikimania
http://etherpad.wikimedia.org/openZIM-Developer-Meeting-2011
/Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l