[Wikitech-l] Phonegap and Wikimedia mobile apps

Could be a pretty good tech. approach. Nevertheless, that means
developing a new app. from scractch... and maintain it.
Emmanuel

Post by Manuel Schneider
Hi,
is this maybe also useful for ZIM - to make ZIM readers which are
working cross-platform?
As far as I understood phonegap is mainly a framework to create mobile
apps based on HTML 5. At least the display of ZIM contents should be
simple then as we just need a HTML widget for that.
But what about libraries needed to read file contents, such as zimlib? I
couldn't find out if Phonegap itself supports native file access (so we
could re-implement ZIM features with that) or if it allows the use of
native libraries.
/Manuel

Manuel Schneider

2011-08-27 09:00:25 UTC

Post by Emmanuel Engelhart
Could be a pretty good tech. approach. Nevertheless, that means
developing a new app. from scractch... and maintain it.

right - but only one app for all mobile platforms. In this case I think
there should be little problem to find a contractor to do it as the
money will be very efficiently spent.

This table looks promising:
* http://www.phonegap.com/about/features

I guess we would need file or storage and network.

/Manuel

--
Regards
Manuel Schneider

Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch

Emmanuel Engelhart

2011-08-27 09:06:23 UTC

Post by Emmanuel Engelhart
Could be a pretty good tech. approach. Nevertheless, that means
developing a new app. from scractch... and maintain it.

right - but only one app for all mobile platforms. In this case I think
there should be little problem to find a contractor to do it as the
money will be very efficiently spent.

It depends what you want to do, how many features you want. You may have
something simple... but having a app. with features like in Kiwix will
be expensive. This also does not fix the problem of the maintenance.

Emmanuel

Christian Pühringer

2011-08-27 22:17:46 UTC

Hi,

It would be really nice if offline (=zim) support was integrated in the
official wikipedia app: The user could switch between online access and
offline reading.

If I understand Tomasz correctly, it is planned to implement the wikipedia app
with phonegap,
so having zim support for phonegap would allow integration of offline support
into the app.
Therefore I think its a good idea to implement zim support for phone gap.

Regarding the technical details there a basically to ways to implement zim
support in phonegap:
- Using phonegap API. Theoretically device independent, but questionable
whether actually working
with sufficient performance on all target platforms.
- As plugin: Implemented as a native component, with bindings to phonegap.

I'd prefer the plugin approach. The additional effort of implementing the zimlib
plugin
for different platforms is not that high. The zimlib (and liblzma) is already
available for
C++ (STL): Symbian[1], Meego[2],
Android (with NDK, cleaner is to use Java),
iPhone (untested, may have some issues (see [5]),
alternative would be to port to objective-C)
and probably Bada
Java (less mature than C++ implementation) : Android, Blackberry and other
J2ME [3] (not sure whether possible on J2ME, but this is even more true for
phonegap API approach)
Porting needs only to be done to:
C#: Windows Mobile
Other: ?
In addition the plugins for the platforms need to be written but this shouldn't
be a too high effort.
The zimlib is already pretty stable, so the maintenance effort for the ports
should not be too bad.

An additional benefit of the plugin-approach is that the ported zimlibs plugins
can also be used for native apps.
(Besides having the plain zimlib, the plugin projects can be used as a starting
point for new projects).

For the phonegap API approach zimlib must be ported to java script. As
mentioned before I doubt that the javascript zimlib would work with sufficient
performance on all (if any) .
devices. See for example [4]

Best regards,
Christian
[1] Neither File (required for phonegap API approach) nor plugins officially
supported in phonegap. However, should be pretty easy to add.
(Either in official (=WRT) phonegap, or in QT port. Probably
better in QT port).
[2] Not supported by phonegap. However, should be possible to use QT phonegap port.
[3] Not supported by phonegap.
[4] http://community.phonegap.com/nitobi/topics/how_to_implement_lzma?from_gsfn=true
[5]
http://stackoverflow.com/questions/823116/how-do-i-use-c-stl-containers-in-my-iphone-app

I haven't used PhoneGap except for some brief testing, but I have used
Titanium Appcelerator, which is another framework in that space, in working
on StatusNet's iPhone& Android app.
Between the two I'd recommend PhoneGap for our usage as preferable over
Titanium, but would appreciate more feedback from people who've done fuller
PhoneGap work.
PhoneGap models around extending a full-screen web view with additional
JavaScript-accessible APIs to use device& OS capabilities (camera, address
book, notifications, etc). This gives you few/no "native widgets" for your
primary screens, but can make it relatively easy to create an HTML/JS-based
web application that's extended with native abilities and can be shipped
into native app stores.
Titanium was originally based on a similar model, but switched to a native
widget bridging system, where your JavaScript code instantiates and
manipulates objects which are bridged to native UI components and such. This
can make your widgets look& feel more native, and can make some UI bits
faster. But it also makes behavior less consistent between platforms; many
widgets or features simply aren't available on all platforms, and last I
checked there was basically *no* working support other than iOS and Android.
(An early BlackBerry demo came out, was insufficient to do anything we
needed, and never got updated that we saw.)
Since the Wikipedia app is mostly a webview and ...... maybe a menu?
PhoneGap is probably a good choice. Titanium can also embed a webview, but
it's a lot more work to deal with two levels of JS! PhoneGap has much
broader device support, but be warned -- it'll use the native webview on
each system, so JS and HTML/CSS support will still vary across platforms.
Debugging in PhoneGap basically devolves to being able to debug a web
application; various tools likehttp://phonegap.github.com/weinre/ can help
with this (or if you code carefully you may get away debugging your app in
your favorite desktop browser directly ;)
Titanium was always a bear to debug things in and basically came down to
'watch the system log output in Android, that's the only place you'll
actually see low-level errors'; this may be better now with their IDE
support.
Titanium also pretty aggressively pushes their support& training services
which I find offputting; their project build tool wants you to login to
their 'cloud' stuff to let you hook up to their remote build& analytics
services, which we didn't ever really use.
Support seemed to center on getting people to take training webinars or
pointing people at the documentation and examples when they ask how to do
something; I didn't find them very responsive about platform bugs or missing
documentation except by contacting their couple of Android developers
one-on-one in IRC to ask for merges -- which was usually a pretty good
experience! Getting fixes for iOS merged was very difficult; I could never
get ahold of their iOS developers directly, and they didn't seem to be any
more responsive to low-level bugs we filed through their customer support
system.
We had to build with a patched version of the iOS and Android runtimes for
quite some time as there were serious bugs. On the plus side, maintaining a
patched branch in git was very easy -- a lot of 'git pull origin master' and
occasionally tidying up conflicts. Their source is all on github and is easy
to fork and not too awful to build, at least for the mobile runtime.
Note that both PhoneGap and Titanium frameworks are open source& hosted on
github, though both require a CLA to submit code upstream. (I have signed
the Titanium CLA to submit patches to them last year; haven't done for
PhoneGap yet.)
-- brion
_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Patrick Reilly

2011-08-28 02:06:18 UTC

That would be a really nice addition to the official application in the future.

We should definitely continue to talk about this and try to figure out
the optimal approach.

Also, once the PhoneGap based Android application is developed it
should be easy to fork and experiment with the various approaches
described below.

— Patrick

Post by Christian PÃ¼hringer
Hi,
It would be really nice if offline (=zim) support was integrated in the
official wikipedia app: The user could switch between online access and
offline reading.
If I understand Tomasz correctly, it is planned to implement the wikipedia app
with phonegap,
so having zim support for phonegap would allow integration of offline support
into the app.
Therefore I think its a good idea to implement zim support for phone gap.
Regarding the technical details there a basically to ways to implement zim
- Using phonegap API. Theoretically device independent, but questionable
whether actually working
with sufficient performance on all target platforms.
- As plugin: Implemented as a native component, with bindings to phonegap.
I'd prefer the plugin approach. The additional effort of implementing the zimlib
plugin
for different platforms is not that high. The zimlib (and liblzma) is already
available for
C++ (STL): Symbian[1], Meego[2],
Android (with NDK, cleaner is to use Java),
iPhone (untested, may have some issues (see [5]),
alternative would be to port to objective-C)
and probably Bada
Java (less mature than C++ implementation) : Android, Blackberry and other
J2ME [3] (not sure whether possible on J2ME, but this is even more true for
phonegap API approach)
C#: Windows Mobile
Other: ?
In addition the plugins for the platforms need to be written but this shouldn't
be a too high effort.
The zimlib is already pretty stable, so the maintenance effort for the ports
should not be too bad.
An additional benefit of the plugin-approach is that the ported zimlibs plugins
can also be used for native apps.
(Besides having the plain zimlib, the plugin projects can be used as a starting
point for new projects).
For the phonegap API approach zimlib must be ported to java script. As
mentioned before I doubt that the javascript zimlib would work with sufficient
performance on all (if any) .
devices. See for example [4]
Best regards,
Christian
[1] Neither File (required for phonegap API approach) nor plugins officially
supported in phonegap. However, should be pretty easy to add.
(Either in official (=WRT) phonegap, or in QT port. Probably
better in QT port).
[2] Not supported by phonegap. However, should be possible to use QT phonegap port.
[3] Not supported by phonegap.
[4] http://community.phonegap.com/nitobi/topics/how_to_implement_lzma?from_gsfn=true
[5]
http://stackoverflow.com/questions/823116/how-do-i-use-c-stl-containers-in-my-iphone-app

I haven't used PhoneGap except for some brief testing, but I have used
Titanium Appcelerator, which is another framework in that space, in working
on StatusNet's iPhone& Android app.
Between the two I'd recommend PhoneGap for our usage as preferable over
Titanium, but would appreciate more feedback from people who've done fuller
PhoneGap work.
PhoneGap models around extending a full-screen web view with additional
JavaScript-accessible APIs to use device& OS capabilities (camera, address
book, notifications, etc). This gives you few/no "native widgets" for your
primary screens, but can make it relatively easy to create an HTML/JS-based
web application that's extended with native abilities and can be shipped
into native app stores.
Titanium was originally based on a similar model, but switched to a native
widget bridging system, where your JavaScript code instantiates and
manipulates objects which are bridged to native UI components and such. This
can make your widgets look& feel more native, and can make some UI bits
faster. But it also makes behavior less consistent between platforms; many
widgets or features simply aren't available on all platforms, and last I
checked there was basically *no* working support other than iOS and Android.
(An early BlackBerry demo came out, was insufficient to do anything we
needed, and never got updated that we saw.)
Since the Wikipedia app is mostly a webview and ...... maybe a menu?
PhoneGap is probably a good choice. Titanium can also embed a webview, but
it's a lot more work to deal with two levels of JS! PhoneGap has much
broader device support, but be warned -- it'll use the native webview on
each system, so JS and HTML/CSS support will still vary across platforms.
Debugging in PhoneGap basically devolves to being able to debug a web
application; various tools likehttp://phonegap.github.com/weinre/ can help
with this (or if you code carefully you may get away debugging your app in
your favorite desktop browser directly ;)
Titanium was always a bear to debug things in and basically came down to
'watch the system log output in Android, that's the only place you'll
actually see low-level errors'; this may be better now with their IDE
support.
Titanium also pretty aggressively pushes their support& training services
which I find offputting; their project build tool wants you to login to
their 'cloud' stuff to let you hook up to their remote build& analytics
services, which we didn't ever really use.
Support seemed to center on getting people to take training webinars or
pointing people at the documentation and examples when they ask how to do
something; I didn't find them very responsive about platform bugs or missing
documentation except by contacting their couple of Android developers
one-on-one in IRC to ask for merges -- which was usually a pretty good
experience! Getting fixes for iOS merged was very difficult; I could never
get ahold of their iOS developers directly, and they didn't seem to be any
more responsive to low-level bugs we filed through their customer support
system.
We had to build with a patched version of the iOS and Android runtimes for
quite some time as there were serious bugs. On the plus side, maintaining a
patched branch in git was very easy -- a lot of 'git pull origin master' and
occasionally tidying up conflicts. Their source is all on github and is easy
to fork and not too awful to build, at least for the mobile runtime.
Note that both PhoneGap and Titanium frameworks are open source& hosted on
github, though both require a CLA to submit code upstream. (I have signed
the Titanium CLA to submit patches to them last year; haven't done for
PhoneGap yet.)
-- brion
_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Tomasz Finc

2011-08-29 10:08:09 UTC

Agreed. I'd LOVE to see openZIM support in a Wikipedia app. We need to
make sure that our users can still access content even if their
disconnected. I personally would love to be able to download a couple
of collections in the openZim format and redistribute as needed. It
would be really cool to download them trivially on a phone and then
redistribute them to areas that may not have a faster internet
connection but have a decent wifi connection.

Christian, Patrick and I can easily connect you with the PhoneGap team
to help you out with this plug in. It would be good to get an early
proof of concept out for the openZim and wikitech developers to play
with. What do you say? Are you up for it?

--tomasz

Post by Patrick Reilly
That would be a really nice addition to the official application in the future.
We should definitely continue to talk about this and try to figure out
the optimal approach.
Also, once the PhoneGap based Android application is developed it
should be easy to fork and experiment with the various approaches
described below.
— Patrick

I haven't used PhoneGap except for some brief testing, but I have used
Titanium Appcelerator, which is another framework in that space, in working
on StatusNet's iPhone& Android app.
Between the two I'd recommend PhoneGap for our usage as preferable over
Titanium, but would appreciate more feedback from people who've done fuller
PhoneGap work.
PhoneGap models around extending a full-screen web view with additional
JavaScript-accessible APIs to use device& OS capabilities (camera, address
book, notifications, etc). This gives you few/no "native widgets" for your
primary screens, but can make it relatively easy to create an HTML/JS-based
web application that's extended with native abilities and can be shipped
into native app stores.
Titanium was originally based on a similar model, but switched to a native
widget bridging system, where your JavaScript code instantiates and
manipulates objects which are bridged to native UI components and such. This
can make your widgets look& feel more native, and can make some UI bits
faster. But it also makes behavior less consistent between platforms; many
widgets or features simply aren't available on all platforms, and last I
checked there was basically *no* working support other than iOS and Android.
(An early BlackBerry demo came out, was insufficient to do anything we
needed, and never got updated that we saw.)
Since the Wikipedia app is mostly a webview and ...... maybe a menu?
PhoneGap is probably a good choice. Titanium can also embed a webview, but
it's a lot more work to deal with two levels of JS! PhoneGap has much
broader device support, but be warned -- it'll use the native webview on
each system, so JS and HTML/CSS support will still vary across platforms.
Debugging in PhoneGap basically devolves to being able to debug a web
application; various tools likehttp://phonegap.github.com/weinre/ can help
with this (or if you code carefully you may get away debugging your app in
your favorite desktop browser directly ;)
Titanium was always a bear to debug things in and basically came down to
'watch the system log output in Android, that's the only place you'll
actually see low-level errors'; this may be better now with their IDE
support.
Titanium also pretty aggressively pushes their support& training services
which I find offputting; their project build tool wants you to login to
their 'cloud' stuff to let you hook up to their remote build& analytics
services, which we didn't ever really use.
Support seemed to center on getting people to take training webinars or
pointing people at the documentation and examples when they ask how to do
something; I didn't find them very responsive about platform bugs or missing
documentation except by contacting their couple of Android developers
one-on-one in IRC to ask for merges -- which was usually a pretty good
experience! Getting fixes for iOS merged was very difficult; I could never
get ahold of their iOS developers directly, and they didn't seem to be any
more responsive to low-level bugs we filed through their customer support
system.
We had to build with a patched version of the iOS and Android runtimes for
quite some time as there were serious bugs. On the plus side, maintaining a
patched branch in git was very easy -- a lot of 'git pull origin master' and
occasionally tidying up conflicts. Their source is all on github and is easy
to fork and not too awful to build, at least for the mobile runtime.
Note that both PhoneGap and Titanium frameworks are open source& hosted on
github, though both require a CLA to submit code upstream. (I have signed
the Titanium CLA to submit patches to them last year; haven't done for
PhoneGap yet.)
-- brion
_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Ted Chien

2011-08-29 11:28:50 UTC

Since you mentioned PhoneGap, how about Titanium Mobile?

http://www.appcelerator.com/products/titanium-mobile-application-development/

It also can help you to build native applications across multiple platforms
(Android and iOS), and you can leverage native C++ code in the application.
It can also leverage file system APIs across multiple platforms.

Regards,
Ted Chien / çŒé¡è
Freelance Software Engineer (Android / Windows)
Secretary, Wikimedia Taiwan
Volunteer, Taipei Google Technology User Group
--
Blog: http://htchien.blogspot.com
LinkedIn: http://linkedin.com/in/htchien
--
Support the Free Knowledge:
http://donate.wikimedia.org
--
Taipei Google Technology User Group
http://www.taipei-gtug.org

-- Sent from my MacBook Pro

Post by Tomasz Finc
Agreed. I'd LOVE to see openZIM support in a Wikipedia app. We need to
make sure that our users can still access content even if their
disconnected. I personally would love to be able to download a couple
of collections in the openZim format and redistribute as needed. It
would be really cool to download them trivially on a phone and then
redistribute them to areas that may not have a faster internet
connection but have a decent wifi connection.
Christian, Patrick and I can easily connect you with the PhoneGap team
to help you out with this plug in. It would be good to get an early
proof of concept out for the openZim and wikitech developers to play
with. What do you say? Are you up for it?
--tomasz

Post by Patrick Reilly
That would be a really nice addition to the official application in the

future.

Post by Patrick Reilly
We should definitely continue to talk about this and try to figure out
the optimal approach.
Also, once the PhoneGap based Android application is developed it
should be easy to fork and experiment with the various approaches
described below.
â Patrick

Post by Christian PÃ¼hringer
Hi,
It would be really nice if offline (=zim) support was integrated in the
official wikipedia app: The user could switch between online access

and

Post by Christian PÃ¼hringer
offline reading.
If I understand Tomasz correctly, it is planned to implement the

wikipedia app

Post by Christian PÃ¼hringer
with phonegap,
so having zim support for phonegap would allow integration of offline

support

Post by Christian PÃ¼hringer
into the app.
Therefore I think its a good idea to implement zim support for phone

gap.

Post by Christian PÃ¼hringer
Regarding the technical details there a basically to ways to implement

zim

Post by Christian PÃ¼hringer
- Using phonegap API. Theoretically device independent, but questionable
whether actually working
with sufficient performance on all target platforms.
- As plugin: Implemented as a native component, with bindings to

phonegap.

Post by Christian PÃ¼hringer
I'd prefer the plugin approach. The additional effort of implementing

the zimlib

Post by Christian PÃ¼hringer
plugin
for different platforms is not that high. The zimlib (and liblzma) is

already

Post by Christian PÃ¼hringer
available for
C++ (STL): Symbian[1], Meego[2],
Android (with NDK, cleaner is to use Java),
iPhone (untested, may have some issues (see

[5]),

Post by Christian PÃ¼hringer
alternative would be to port to objective-C)
and probably Bada
Java (less mature than C++ implementation) : Android, Blackberry

and other

Post by Christian PÃ¼hringer
J2ME [3] (not sure whether possible on J2ME, but this is even more true

for

Post by Christian PÃ¼hringer
phonegap API approach)
C#: Windows Mobile
Other: ?
In addition the plugins for the platforms need to be written but this

shouldn't

Post by Christian PÃ¼hringer
be a too high effort.
The zimlib is already pretty stable, so the maintenance effort for the

ports

Post by Christian PÃ¼hringer
should not be too bad.
An additional benefit of the plugin-approach is that the ported zimlibs

plugins

Post by Christian PÃ¼hringer
can also be used for native apps.
(Besides having the plain zimlib, the plugin projects can be used as a

starting

Post by Christian PÃ¼hringer
point for new projects).
For the phonegap API approach zimlib must be ported to java script. As
mentioned before I doubt that the javascript zimlib would work with

sufficient

Post by Christian PÃ¼hringer
performance on all (if any) .
devices. See for example [4]
Best regards,
Christian
[1] Neither File (required for phonegap API approach) nor plugins

officially

Post by Christian PÃ¼hringer
supported in phonegap. However, should be pretty easy to add.
(Either in official (=WRT) phonegap, or in QT port.

Probably

Post by Christian PÃ¼hringer
better in QT port).
[2] Not supported by phonegap. However, should be possible to use QT

phonegap port.

Post by Christian PÃ¼hringer
[3] Not supported by phonegap.
[4]

http://community.phonegap.com/nitobi/topics/how_to_implement_lzma?from_gsfn=true

Post by Christian PÃ¼hringer
[5]

http://stackoverflow.com/questions/823116/how-do-i-use-c-stl-containers-in-my-iphone-app

Post by Manuel Schneider
couldn't find out if Phonegap itself supports native file access (so we
could re-implement ZIM features with that) or if it allows the use of
native libraries.
/Manuel

I've been asking around on IRC but thought it would be good to open

to a larger audience.
Has anyone here used PhoneGap (http://www.phonegap.com/) for mobile
app development? I'm eager to get your thoughts and potentially
brainstorm some new ideas.

I haven't used PhoneGap except for some brief testing, but I have

used

Titanium Appcelerator, which is another framework in that space, in

working

on StatusNet's iPhone& Android app.
Between the two I'd recommend PhoneGap for our usage as preferable

over

Titanium, but would appreciate more feedback from people who've done

fuller

PhoneGap work.
PhoneGap models around extending a full-screen web view with

additional

JavaScript-accessible APIs to use device& OS capabilities (camera,

address

book, notifications, etc). This gives you few/no "native widgets" for

your

primary screens, but can make it relatively easy to create an

HTML/JS-based

web application that's extended with native abilities and can be

shipped

into native app stores.
Titanium was originally based on a similar model, but switched to a

native

widget bridging system, where your JavaScript code instantiates and
manipulates objects which are bridged to native UI components and

such. This

can make your widgets look& feel more native, and can make some UI

bits

faster. But it also makes behavior less consistent between platforms;

many

widgets or features simply aren't available on all platforms, and

last I

checked there was basically *no* working support other than iOS and

Android.

(An early BlackBerry demo came out, was insufficient to do anything

needed, and never got updated that we saw.)
Since the Wikipedia app is mostly a webview and ...... maybe a menu?
PhoneGap is probably a good choice. Titanium can also embed a

webview, but

it's a lot more work to deal with two levels of JS! PhoneGap has much
broader device support, but be warned -- it'll use the native webview

each system, so JS and HTML/CSS support will still vary across

platforms.

Debugging in PhoneGap basically devolves to being able to debug a web
application; various tools likehttp://phonegap.github.com/weinre/ can help
with this (or if you code carefully you may get away debugging your

app in

your favorite desktop browser directly ;)
Titanium was always a bear to debug things in and basically came down

'watch the system log output in Android, that's the only place you'll
actually see low-level errors'; this may be better now with their IDE
support.
Titanium also pretty aggressively pushes their support& training

services

which I find offputting; their project build tool wants you to login

their 'cloud' stuff to let you hook up to their remote build&

analytics

services, which we didn't ever really use.
Support seemed to center on getting people to take training webinars

pointing people at the documentation and examples when they ask how

to do

something; I didn't find them very responsive about platform bugs or

missing

documentation except by contacting their couple of Android developers
one-on-one in IRC to ask for merges -- which was usually a pretty

good

experience! Getting fixes for iOS merged was very difficult; I could

never

get ahold of their iOS developers directly, and they didn't seem to

be any

more responsive to low-level bugs we filed through their customer

support

system.
We had to build with a patched version of the iOS and Android

runtimes for

quite some time as there were serious bugs. On the plus side,

maintaining a

patched branch in git was very easy -- a lot of 'git pull origin

master' and

occasionally tidying up conflicts. Their source is all on github and

is easy

to fork and not too awful to build, at least for the mobile runtime.
Note that both PhoneGap and Titanium frameworks are open source&

hosted on

github, though both require a CLA to submit code upstream. (I have

signed

the Titanium CLA to submit patches to them last year; haven't done

for

PhoneGap yet.)
-- brion
_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

Tomasz Finc

2011-08-29 17:13:27 UTC

Ted, read Brions response about Titanium on 8/13 to see why were
opting to experiment with PhoneGap first.

That being said .. if you want to write a Titanium version with the
same features then go for it. We can then benchmark them for ease of
development and overall speed to see which is best across all
platforms.

--tomasz

Since you mentioned PhoneGap, how about Titanium Mobile?
http://www.appcelerator.com/products/titanium-mobile-application-development/
It also can help you to build native applications across multiple platforms
(Android and iOS), and you can leverage native C++ code in the application.
It can also leverage file system APIs across multiple platforms.
Regards,
Ted Chien / 眼鏡虎
Freelance Software Engineer (Android / Windows)
Secretary, Wikimedia Taiwan
Volunteer, Taipei Google Technology User Group
--
Blog: http://htchien.blogspot.com
LinkedIn: http://linkedin.com/in/htchien
--
http://donate.wikimedia.org
--
Taipei Google Technology User Group
http://www.taipei-gtug.org
-- Sent from my MacBook Pro

Post by Christian PÃ¼hringer
Hi,
It would be really nice if offline (=zim) support was integrated in the
official wikipedia app: The user could switch between online access and
offline reading.
If I understand Tomasz correctly, it is planned to implement the wikipedia app
with phonegap,
so having zim support for phonegap would allow integration of offline support
into the app.
Therefore I think its a good idea to implement zim support for phone gap.
Regarding the technical details there a basically to ways to implement zim
- Using phonegap API. Theoretically device independent, but questionable
whether actually working
with sufficient performance on all target platforms.
- As plugin: Implemented as a native component, with bindings to phonegap.
I'd prefer the plugin approach. The additional effort of implementing the zimlib
plugin
for different platforms is not that high. The zimlib (and liblzma) is already
available for
C++ (STL): Symbian[1], Meego[2],
Android (with NDK, cleaner is to use Java),
iPhone (untested, may have some issues (see
[5]),
alternative would be to port to objective-C)
and probably Bada
Java (less mature than C++ implementation) : Android, Blackberry and other
J2ME [3] (not sure whether possible on J2ME, but this is even more true for
phonegap API approach)
C#: Windows Mobile
Other: ?
In addition the plugins for the platforms need to be written but this shouldn't
be a too high effort.
The zimlib is already pretty stable, so the maintenance effort for the ports
should not be too bad.
An additional benefit of the plugin-approach is that the ported zimlibs plugins
can also be used for native apps.
(Besides having the plain zimlib, the plugin projects can be used as a starting
point for new projects).
For the phonegap API approach zimlib must be ported to java script. As
mentioned before I doubt that the javascript zimlib would work with sufficient
performance on all (if any) .
devices. See for example [4]
Best regards,
Christian
[1] Neither File (required for phonegap API approach) nor plugins officially
supported in phonegap. However, should be pretty easy to add.
(Either in official (=WRT) phonegap, or in QT port. Probably
better in QT port).
[2] Not supported by phonegap. However, should be possible to use QT phonegap port.
[3] Not supported by phonegap.
[4]
http://community.phonegap.com/nitobi/topics/how_to_implement_lzma?from_gsfn=true
[5]
http://stackoverflow.com/questions/823116/how-do-i-use-c-stl-containers-in-my-iphone-app

I haven't used PhoneGap except for some brief testing, but I have used
Titanium Appcelerator, which is another framework in that space, in working
on StatusNet's iPhone& Android app.
Between the two I'd recommend PhoneGap for our usage as preferable over
Titanium, but would appreciate more feedback from people who've done fuller
PhoneGap work.
PhoneGap models around extending a full-screen web view with additional
JavaScript-accessible APIs to use device& OS capabilities (camera, address
book, notifications, etc). This gives you few/no "native widgets" for your
primary screens, but can make it relatively easy to create an HTML/JS-based
web application that's extended with native abilities and can be shipped
into native app stores.
Titanium was originally based on a similar model, but switched to a native
widget bridging system, where your JavaScript code instantiates and
manipulates objects which are bridged to native UI components and such. This
can make your widgets look& feel more native, and can make some UI bits
faster. But it also makes behavior less consistent between platforms; many
widgets or features simply aren't available on all platforms, and last I
checked there was basically *no* working support other than iOS and Android.
(An early BlackBerry demo came out, was insufficient to do anything we
needed, and never got updated that we saw.)
Since the Wikipedia app is mostly a webview and ...... maybe a menu?
PhoneGap is probably a good choice. Titanium can also embed a webview, but
it's a lot more work to deal with two levels of JS! PhoneGap has much
broader device support, but be warned -- it'll use the native webview on
each system, so JS and HTML/CSS support will still vary across platforms.
Debugging in PhoneGap basically devolves to being able to debug a web
application; various tools likehttp://phonegap.github.com/weinre/ can help
with this (or if you code carefully you may get away debugging your app in
your favorite desktop browser directly ;)
Titanium was always a bear to debug things in and basically came down to
'watch the system log output in Android, that's the only place you'll
actually see low-level errors'; this may be better now with their IDE
support.
Titanium also pretty aggressively pushes their support& training services
which I find offputting; their project build tool wants you to login to
their 'cloud' stuff to let you hook up to their remote build& analytics
services, which we didn't ever really use.
Support seemed to center on getting people to take training webinars or
pointing people at the documentation and examples when they ask how to do
something; I didn't find them very responsive about platform bugs or missing
documentation except by contacting their couple of Android developers
one-on-one in IRC to ask for merges -- which was usually a pretty good
experience! Getting fixes for iOS merged was very difficult; I could never
get ahold of their iOS developers directly, and they didn't seem to be any
more responsive to low-level bugs we filed through their customer support
system.
We had to build with a patched version of the iOS and Android runtimes for
quite some time as there were serious bugs. On the plus side, maintaining a
patched branch in git was very easy -- a lot of 'git pull origin master' and
occasionally tidying up conflicts. Their source is all on github and is easy
to fork and not too awful to build, at least for the mobile runtime.
Note that both PhoneGap and Titanium frameworks are open source& hosted on
github, though both require a CLA to submit code upstream. (I have signed
the Titanium CLA to submit patches to them last year; haven't done for
PhoneGap yet.)
-- brion
_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

Christian Pühringer

2011-09-02 21:11:02 UTC

Hi Tomasz,

I cannot promise, but I'll try to find the time to implement some proof of
concept for zim+phonegap.
(The problem is that this is not that low effort, because all potential
platforms either don't have
tested zimlib support or don't have phonegap plugin support out-of-the-box.)

For the proof-of-concept I expect that I don't need support by phonegap, but
mid-term it would make sense
if you could connect me with them (e.g. regarding plugin support for
symbian/meego)).

Also please keep me updated about the state of the Wikipedia app for phonegap.
(I assume there is nothing available right now?) .

Best regards,
Christian

I haven't used PhoneGap except for some brief testing, but I have used
Titanium Appcelerator, which is another framework in that space, in working
on StatusNet's iPhone& Android app.
Between the two I'd recommend PhoneGap for our usage as preferable over
Titanium, but would appreciate more feedback from people who've done fuller
PhoneGap work.
PhoneGap models around extending a full-screen web view with additional
JavaScript-accessible APIs to use device& OS capabilities (camera, address
book, notifications, etc). This gives you few/no "native widgets" for your
primary screens, but can make it relatively easy to create an HTML/JS-based
web application that's extended with native abilities and can be shipped
into native app stores.
Titanium was originally based on a similar model, but switched to a native
widget bridging system, where your JavaScript code instantiates and
manipulates objects which are bridged to native UI components and such. This
can make your widgets look& feel more native, and can make some UI bits
faster. But it also makes behavior less consistent between platforms; many
widgets or features simply aren't available on all platforms, and last I
checked there was basically *no* working support other than iOS and Android.
(An early BlackBerry demo came out, was insufficient to do anything we
needed, and never got updated that we saw.)
Since the Wikipedia app is mostly a webview and ...... maybe a menu?
PhoneGap is probably a good choice. Titanium can also embed a webview, but
it's a lot more work to deal with two levels of JS! PhoneGap has much
broader device support, but be warned -- it'll use the native webview on
each system, so JS and HTML/CSS support will still vary across platforms.
Debugging in PhoneGap basically devolves to being able to debug a web
application; various tools likehttp://phonegap.github.com/weinre/ can help
with this (or if you code carefully you may get away debugging your app in
your favorite desktop browser directly ;)
Titanium was always a bear to debug things in and basically came down to
'watch the system log output in Android, that's the only place you'll
actually see low-level errors'; this may be better now with their IDE
support.
Titanium also pretty aggressively pushes their support& training services
which I find offputting; their project build tool wants you to login to
their 'cloud' stuff to let you hook up to their remote build& analytics
services, which we didn't ever really use.
Support seemed to center on getting people to take training webinars or
pointing people at the documentation and examples when they ask how to do
something; I didn't find them very responsive about platform bugs or missing
documentation except by contacting their couple of Android developers
one-on-one in IRC to ask for merges -- which was usually a pretty good
experience! Getting fixes for iOS merged was very difficult; I could never
get ahold of their iOS developers directly, and they didn't seem to be any
more responsive to low-level bugs we filed through their customer support
system.
We had to build with a patched version of the iOS and Android runtimes for
quite some time as there were serious bugs. On the plus side, maintaining a
patched branch in git was very easy -- a lot of 'git pull origin master' and
occasionally tidying up conflicts. Their source is all on github and is easy
to fork and not too awful to build, at least for the mobile runtime.
Note that both PhoneGap and Titanium frameworks are open source& hosted on
github, though both require a CLA to submit code upstream. (I have signed
the Titanium CLA to submit patches to them last year; haven't done for
PhoneGap yet.)
-- brion
_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

Tomasz Finc

2011-09-14 21:33:47 UTC

CC'ing Herman Wong from Nitobi who's helping develop an Android
version of the Wikimedia iOS app. I'll be sending an email with more
detail on that later this week.

Herman, Christian is looking to write a plug in to implement a simpler
reader for the http://openzim.org/ standard.

Christian, come join us on IRC #wikimedia-mobile with whatever
questions you might have.

--tomasz

Post by Christian PÃ¼hringer
Hi Tomasz,
I cannot promise, but I'll try to find the time to implement some proof of
concept for zim+phonegap.
(The problem is that this is not that low effort, because all potential
platforms either don't have
tested zimlib support or don't have phonegap plugin support out-of-the-box.)
For the proof-of-concept I expect that I don't need support by phonegap, but
mid-term it would make sense
if you could connect me with them (e.g. regarding plugin support for
symbian/meego)).
Also please keep me updated about the state of the Wikipedia app for phonegap.
(I assume there is nothing available right now?) .
Best regards,
Christian

Post by Christian PÃ¼hringer
Hi,
It would be really nice if offline (=zim) support was integrated in the
official wikipedia app: The user could switch between online access and
offline reading.
If I understand Tomasz correctly, it is planned to implement the wikipedia app
with phonegap,
so having zim support for phonegap would allow integration of offline support
into the app.
Therefore I think its a good idea to implement zim support for phone gap.
Regarding the technical details there a basically to ways to implement zim
- Using phonegap API. Theoretically device independent, but questionable
whether actually working
with sufficient performance on all target platforms.
- As plugin: Implemented as a native component, with bindings to phonegap.
I'd prefer the plugin approach. The additional effort of implementing the zimlib
plugin
for different platforms is not that high. The zimlib (and liblzma) is already
available for
C++ (STL): Symbian[1], Meego[2],
Android (with NDK, cleaner is to use Java),
iPhone (untested, may have some issues (see [5]),
alternative would be to port to objective-C)
and probably Bada
Java (less mature than C++ implementation) : Android, Blackberry and other
J2ME [3] (not sure whether possible on J2ME, but this is even more true for
phonegap API approach)
C#: Windows Mobile
Other: ?
In addition the plugins for the platforms need to be written but this shouldn't
be a too high effort.
The zimlib is already pretty stable, so the maintenance effort for the ports
should not be too bad.
An additional benefit of the plugin-approach is that the ported zimlibs plugins
can also be used for native apps.
(Besides having the plain zimlib, the plugin projects can be used as a starting
point for new projects).
For the phonegap API approach zimlib must be ported to java script. As
mentioned before I doubt that the javascript zimlib would work with sufficient
performance on all (if any) .
devices. See for example [4]
Best regards,
Christian
[1] Neither File (required for phonegap API approach) nor plugins officially
supported in phonegap. However, should be pretty easy to add.
(Either in official (=WRT) phonegap, or in QT port. Probably
better in QT port).
[2] Not supported by phonegap. However, should be possible to use QT phonegap port.
[3] Not supported by phonegap.
[4]
http://community.phonegap.com/nitobi/topics/how_to_implement_lzma?from_gsfn=true
[5]
http://stackoverflow.com/questions/823116/how-do-i-use-c-stl-containers-in-my-iphone-app

I haven't used PhoneGap except for some brief testing, but I have used
Titanium Appcelerator, which is another framework in that space, in working
on StatusNet's iPhone& Android app.
Between the two I'd recommend PhoneGap for our usage as preferable over
Titanium, but would appreciate more feedback from people who've done fuller
PhoneGap work.
PhoneGap models around extending a full-screen web view with additional
JavaScript-accessible APIs to use device& OS capabilities (camera,
address
book, notifications, etc). This gives you few/no "native widgets" for your
primary screens, but can make it relatively easy to create an HTML/JS-based
web application that's extended with native abilities and can be shipped
into native app stores.
Titanium was originally based on a similar model, but switched to a native
widget bridging system, where your JavaScript code instantiates and
manipulates objects which are bridged to native UI components and such. This
can make your widgets look& feel more native, and can make some UI
bits
faster. But it also makes behavior less consistent between platforms; many
widgets or features simply aren't available on all platforms, and last I
checked there was basically *no* working support other than iOS and Android.
(An early BlackBerry demo came out, was insufficient to do anything we
needed, and never got updated that we saw.)
Since the Wikipedia app is mostly a webview and ...... maybe a menu?
PhoneGap is probably a good choice. Titanium can also embed a webview, but
it's a lot more work to deal with two levels of JS! PhoneGap has much
broader device support, but be warned -- it'll use the native webview on
each system, so JS and HTML/CSS support will still vary across platforms.
Debugging in PhoneGap basically devolves to being able to debug a web
application; various tools likehttp://phonegap.github.com/weinre/ can help
with this (or if you code carefully you may get away debugging your app in
your favorite desktop browser directly ;)
Titanium was always a bear to debug things in and basically came down to
'watch the system log output in Android, that's the only place you'll
actually see low-level errors'; this may be better now with their IDE
support.
Titanium also pretty aggressively pushes their support& training
services
which I find offputting; their project build tool wants you to login to
their 'cloud' stuff to let you hook up to their remote build& analytics
services, which we didn't ever really use.
Support seemed to center on getting people to take training webinars or
pointing people at the documentation and examples when they ask how to do
something; I didn't find them very responsive about platform bugs or missing
documentation except by contacting their couple of Android developers
one-on-one in IRC to ask for merges -- which was usually a pretty good
experience! Getting fixes for iOS merged was very difficult; I could never
get ahold of their iOS developers directly, and they didn't seem to be any
more responsive to low-level bugs we filed through their customer support
system.
We had to build with a patched version of the iOS and Android runtimes for
quite some time as there were serious bugs. On the plus side, maintaining a
patched branch in git was very easy -- a lot of 'git pull origin master' and
occasionally tidying up conflicts. Their source is all on github and is easy
to fork and not too awful to build, at least for the mobile runtime.
Note that both PhoneGap and Titanium frameworks are open source& hosted on
github, though both require a CLA to submit code upstream. (I have signed
the Titanium CLA to submit patches to them last year; haven't done for
PhoneGap yet.)
-- brion
_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
Wikitech-l mailing list
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

Brion Vibber

2011-09-14 22:09:07 UTC

Post by Tomasz Finc
CC'ing Herman Wong from Nitobi who's helping develop an Android
version of the Wikimedia iOS app. I'll be sending an email with more
detail on that later this week.
Herman, Christian is looking to write a plug in to implement a simpler
reader for the http://openzim.org/ standard.

As I understand, for Android this will roughly require:
* get zimreader-java classes running under Android+Darwin <
http://svn.openzim.org/viewvc.cgi/trunk/zimreader-java/>
* build an Android PhoneGap Java plugin wrapping it <
http://wiki.phonegap.com/w/page/36753494/How%20to%20Create%20a%20PhoneGap%20Plugin%20for%20Android
iPhone/iPad will require:
* build C++ zimlib and its dependencies for iOS (cross-compiling from Mac OS
X)
* build an Obj-C iOS a PhoneGap plugin wrapping it <
http://wiki.phonegap.com/w/page/36753496/How%20to%20Create%20a%20PhoneGap%20Plugin%20for%20iOS
I suspect the Android one will be easier, but you never know. ;)

The Java bits ... may also be adaptable to BlackBerry:
http://wiki.phonegap.com/w/page/35799737/How-To-Create-a-PhoneGap-Plugin-for-BlackBerry-WebWorks

No other platform seems to have PhoneGap plugin documentation that I can
find.

Alternately, different code could be written to implement the file format
reader directly in the PhoneGap plugins if that ends up easier than porting
the other libraries; the most performance-critical part is probably the LZMA
decompression, which could be a smaller more isolated library.

http://openzim.org/ZIM_File_Format

-- brion

Christian Pühringer

2011-09-24 12:24:17 UTC

Hi,

I've some first results regarding the feasibility of phonegap+zim on android.
While it basically works to load an article (without images) and display it
(thanks to Arunesh for doing the java port),
I've encountered some major challenges:
1. How to display images?
It is apparently quite tricky to display images which are not either internet
links or local files in a webview on android:
The most promising concepts I found are:
a). Extract images from zim to local file system and provide them over a
ContentProvider.
b) Replace image src with base64 encoded image data. (Either directly on loading
of article or by using java script)
I'll try to implement at least one of this approaches to find out whether it is
feasible to do so.
However, obviously both approaches are not really perfect solutions, so if
anybody has a better idea please let me know.

Note that there actually would be a good way to do it
(WebViewClient.shouldInterceptRequest()) but unfortunately this is API-level 11
which means
that there is no android mobile phone currently available in the market which
supports it. Note that additionally it would not be possible
to use this with current phonegap versions. Therefore it would be nice if in
future phonegap versions this functionality is provided to plugins. While
this would not help for current mobile phones, it would allow a more efficient
implementation of zim support in the future.

2. Performance
The JAVA liblzma performance is pretty bad: To increase efficiency of
compression in the zim-format articles (and also all
other data like images) are stored in clusters. Cluster size is apparently about
1 MB. This implies that loading an article
which is stored at the end of a cluster involves decompressing the complete
cluster. While this is not a problem with C code
even on embedded devices, it seems to be a problem with Java: Reading an article
at the end of a cluster takes close to 20 s on
my test android phone. As the phone is pretty low end (Orange
Boston ) and uses an old android version (Eclair) without a just-in-timer
compiler I expect that other models
are significantly faster. However, I doubt that the performance gain will be
sufficient to bring article load time in a range of << 1 s.
I am going to try it out, but I'd expect that we probably have to switch to
native code for zimlib support. (At least for liblzma).
An other approach is to reduce cluster size of zim files. I am not sure right
now whether this would be sufficiently fast, but it is
worth considering it as an option: While for android being able to use the java
implementation is a benefit, it is also not a big thing
if native code has to be used. However, more concerning is that it may not be
possible to support for Windows Mobile at all with the
current cluster size. (Because AFAIK not native code is supported)

Note that both issues are not only related to phonegap, they would basically
also affect a regular android app. (1. only
if webview is used, which is however the by far simplest approach, except for
the image issue)

Best regards,
Christian

Emmanuel Engelhart

2011-09-24 12:30:56 UTC

Post by Christian PÃ¼hringer
The JAVA liblzma performance is pretty bad: To increase efficiency of
compression in the zim-format articles (and also all
other data like images) are stored in clusters. Cluster size is apparently about
1 MB. This implies that loading an article
which is stored at the end of a cluster involves decompressing the complete
cluster.

Arunesh Mathur

2011-09-24 12:42:03 UTC

Yes, images are not compressed in ZIM files, as Emmanuel pointed out.

Also, I have tried decompressing a LZMA compressed file on an Android phone
(HTC Wildfire to be precise), and the decompression speed is not a problem.

LZMA-Java is under frequent development by Lasse Collin, so we should make
sure that the latest code is used.

On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel Engelhart

apparently about

Post by Christian PÃ¼hringer
1 MB. This implies that loading an article
which is stored at the end of a cluster involves decompressing the

complete

--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.

Christian Pühringer

2011-10-03 17:40:54 UTC

Hi Arunesh,

Firstly, thank you again for doing the java port.

I've one question regarding the features of the port:
Is there or do you plan to add a feature for iterating over the index? (Similar
to find/findByTitle in C++ zimlib)
This would be required to add some auto complete feature to the phonegap app.

Best regards,
Christian

Post by Arunesh Mathur
Yes, images are not compressed in ZIM files, as Emmanuel pointed out.
Also, I have tried decompressing a LZMA compressed file on an Android phone
(HTC Wildfire to be precise), and the decompression speed is not a problem.
LZMA-Java is under frequent development by Lasse Collin, so we should make
sure that the latest code is used.

apparently about

Post by Christian PÃ¼hringer
1 MB. This implies that loading an article
which is stored at the end of a cluster involves decompressing the complete
cluster.

Images should not be compressed in ZIM files for the obvious reasons
they mainly are already compressed. This is the case for all ZIM files I
made. As far as I know this is also the case for Mediawiki:Collection
build ZIM files.
Emmanuel
_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l
--
Best,
Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.
_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

Arunesh Mathur

2011-10-05 06:22:06 UTC

Hi Christian,
You can look at getDirectoryInfo(String articleName, char
namespace) in ZimReader.java
This'll return null if there is the article doesn't exist, else the
DirectoryEntry object. (Is this what you were looking for?)

Actually I plan on adding more features if required. Do you think there are
any missing, when compared to zimlib? If yes, could you list them?

Have you tested zimreader-java on an Android phone? How is it's performance?

Post by Christian PÃ¼hringer
Hi Arunesh,
Firstly, thank you again for doing the java port.
Is there or do you plan to add a feature for iterating over the index?
(Similar to find/findByTitle in C++ zimlib)
This would be required to add some auto complete feature to the phonegap app.
Best regards,
Christian
Yes, images are not compressed in ZIM files, as Emmanuel pointed out.
Also, I have tried decompressing a LZMA compressed file on an Android phone
(HTC Wildfire to be precise), and the decompression speed is not a problem.
LZMA-Java is under frequent development by Lasse Collin, so we should make
sure that the latest code is used.
On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel Engelhart <

apparently about

Post by Christian PÃ¼hringer
1 MB. This implies that loading an article
which is stored at the end of a cluster involves decompressing the

complete

--
Best,
Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.
_______________________________________________

--
Best,

Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.

Christian Pühringer

2011-10-06 21:01:02 UTC

Hi Arunesh,

I'd need something like findByTitle(namespace, title) in zimlib: It returns an
iterator pointing
to the lexicographically next article. I'd like to use this to implement an
auto suggest
feature (Same as in online wikipedia search box).
Note: While for auto suggest iterating forward is necessary, for other features
it would be good if it is also
possible to iterate backward (same as in zimlib).

One other thing which may be currently missing is support for title and url
search/get article
functionality. It looks to me the right now everything operates on urls and not
on titles.
For searching it is necessary to also have title based functions. (Like
findByTitle).
Note that while currently in most zimfiles urls have the form
'Namespace'/'Title', this is not guaranteed,
the url could be completely different from the title.

Regarding performance, have you missed Brion's and my results posted in this
thread?
(They may be only in the wikitech-l-RusutVdil2icGmH+5r0DM0B+***@public.gmane.org and not in the
dev-l-***@public.gmane.org list)

Best regards,
Christian

Post by Arunesh Mathur
Hi Christian,
You can look at getDirectoryInfo(String articleName, char
namespace) in ZimReader.java
This'll return null if there is the article doesn't exist, else the
DirectoryEntry object. (Is this what you were looking for?)
Actually I plan on adding more features if required. Do you think there are
any missing, when compared to zimlib? If yes, could you list them?
Have you tested zimreader-java on an Android phone? How is it's performance?
Hi Arunesh,
Firstly, thank you again for doing the java port.
Is there or do you plan to add a feature for iterating over the index?
(Similar to find/findByTitle in C++ zimlib)
This would be required to add some auto complete feature to the phonegap app.
Best regards,
Christian

Post by Arunesh Mathur
Yes, images are not compressed in ZIM files, as Emmanuel pointed out.
Also, I have tried decompressing a LZMA compressed file on an Android
phone (HTC Wildfire to be precise), and the decompression speed is not a
problem.
LZMA-Java is under frequent development by Lasse Collin, so we should
make sure that the latest code is used.
On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel Engelhart

apparently about

Post by Christian PÃ¼hringer
1 MB. This implies that loading an article
which is stored at the end of a cluster involves decompressing the

complete

--
Best,
Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.

Arunesh Mathur

2011-10-07 10:39:41 UTC

Hi Christian,
Sure, that seems easy enough to implement. I'll get that
feature up and running soon.

Yes, most of the functions are based on title, I did that because I didn't
find any difference between the two when I browsed around 5-6 ZIM files. But
as you say, I'll add their URL counterparts as well. I'll also add the
documentation-gen module to the repository.

Hmm, I'm not subscribed to that mailing list. I'll go through the thread
that discusses performance.

-Arunesh Mathur

Post by Christian PÃ¼hringer
Hi Arunesh,
I'd need something like findByTitle(namespace, title) in zimlib: It
returns an iterator pointing
to the lexicographically next article. I'd like to use this to implement
an auto suggest
feature (Same as in online wikipedia search box).
Note: While for auto suggest iterating forward is necessary, for other
features it would be good if it is also
possible to iterate backward (same as in zimlib).
One other thing which may be currently missing is support for title and url
search/get article
functionality. It looks to me the right now everything operates on urls and
not on titles.
For searching it is necessary to also have title based functions. (Like
findByTitle).
Note that while currently in most zimfiles urls have the form
'Namespace'/'Title', this is not guaranteed,
the url could be completely different from the title.
Regarding performance, have you missed Brion's and my results posted in
this thread?
Best regards,
Christian
Hi Christian,
You can look at getDirectoryInfo(String articleName,
char namespace) in ZimReader.java
This'll return null if there is the article doesn't exist, else the
DirectoryEntry object. (Is this what you were looking for?)
Actually I plan on adding more features if required. Do you think there are
any missing, when compared to zimlib? If yes, could you list them?
Have you tested zimreader-java on an Android phone? How is it's performance?

Post by Christian PÃ¼hringer
Hi Arunesh,
Firstly, thank you again for doing the java port.
Is there or do you plan to add a feature for iterating over the index?
(Similar to find/findByTitle in C++ zimlib)
This would be required to add some auto complete feature to the phonegap app.
Best regards,
Christian
Yes, images are not compressed in ZIM files, as Emmanuel pointed out.
Also, I have tried decompressing a LZMA compressed file on an Android
phone (HTC Wildfire to be precise), and the decompression speed is not a
problem.
LZMA-Java is under frequent development by Lasse Collin, so we should make
sure that the latest code is used.
On Sat, Sep 24, 2011 at 6:00 PM, Emmanuel Engelhart <

apparently about

Post by Christian PÃ¼hringer
1 MB. This implies that loading an article
which is stored at the end of a cluster involves decompressing the

complete

--
Best,
Arunesh Mathur
IV year, Undergraduate,
Department of Computer Science and Engineering,
National Institute of Technology Karnataka Surathkal,
India.

Christian Pühringer

2011-09-24 12:49:35 UTC

Hi Emanuel,

Makes sense that images are not compressed in zim file again, thanks for the
clarification.
In particular for windows mobile this increases the probability that with
reducing the cluster size
a sufficient performance is achieved (i.p. if there was a better way to display
images than in android, but I haven't looked into this).
For android I still expect that it it will be better to use a native library.

Christian

Tomasz Finc

2011-09-24 21:14:31 UTC

Awesome work Christian. Where can we find your code to test out?

If you don't have a place then I'll carve one out in our depots.

Post by Christian PÃ¼hringer
Hi Emanuel,
Makes sense that images are not compressed in zim file again, thanks for the
clarification.
In particular for windows mobile this increases the probability that with
reducing the cluster size
a sufficient performance is achieved (i.p. if there was a better way to display
images than in android, but I haven't looked into this).
For android I still expect that it it will be better to use a native library.
Christian

_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

Christian Pühringer

2011-09-27 20:51:36 UTC

Hi Tomasz,

I've pushed the code to https://github.com/cip/zimgap-android.
Note that the only functionality is currently to load an article from the zim
file and display it.
In particular there is no search functionality, so you have to know the exact
article name to open it.
The reason is simply that I am currently focusing on the feasibility, and I
expect that the main issues are related to loading
an article and and not related to searching the index.

Best regards,
Christian

Post by Tomasz Finc
Awesome work Christian. Where can we find your code to test out?
If you don't have a place then I'll carve one out in our depots.

about

Post by Christian PÃ¼hringer
1 MB. This implies that loading an article
which is stored at the end of a cluster involves decompressing the complete
cluster.

_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

Tomasz Finc

2011-09-27 21:00:15 UTC

Awesome .. this will be fun to play with and a great addition to the
Wikipedia app community.

--tomasz

Post by Christian PÃ¼hringer
Hi Tomasz,
I've pushed the code to https://github.com/cip/zimgap-android.
Note that the only functionality is currently to load an article from the
zim file and display it.
In particular there is no search functionality, so you have to know the
exact article name to open it.
The reason is simply that I am currently focusing on the feasibility, and I
expect that the main issues are related to loading
an article and and not related to searching the index.
Best regards,
Christian
Awesome work Christian. Where can we find your code to test out?
If you don't have a place then I'll carve one out in our depots.

_______________________________________________
dev-l mailing list
https://intern.openzim.org/mailman/listinfo/dev-l

Brion Vibber

2011-09-28 01:01:30 UTC

Post by Christian PÃ¼hringer
I've pushed the code to https://github.com/cip/zimgap-android.
Note that the only functionality is currently to load an article from the zim
file and display it.
In particular there is no search functionality, so you have to know the exact
article name to open it.
The reason is simply that I am currently focusing on the feasibility, and I
expect that the main issues are related to loading
an article and and not related to searching the index.

Just to warn: run 'git submodule init && git submodule update' after cloning
the repository, or Eclipse will whinge about being unable to find the
xz-java sources. :)

I did a quick test on my Nexus One (running Android 2.3.6, so with JIT).
There are some faster processors out there on higher-end phones, but there's
also a lot of cheaper phones that aren't going to be any faster than this.

The wikipedia-de.zim test file hasn't downloaded here yet so I started with
a smaller file generated from:
http://en.wikipedia.org/wiki/User:Brion_VIBBER/Books/SmallZimTest

Fastest of three runs each:

'Barack Obama' (a fairly long article with a few hundred KB of HTML)
Load time 5298ms
Render time 439ms

'Husiatyn Raion' (a shorter article)
Load time 1789ms
Render time 23ms

Better than 20 seconds, but still pretty slow; I'm not sure offhand how the
articles get placed in their blocks so there may well be worse worst cases.
:(

Definitely worth trying the native-code library at least for the LZMA --
though in a worst case, a few seconds to load isn't worse than you'll have
on a slow mobile data network. :) Make sure it runs on a background thread
and shows a spinner or progress bar if we do end up having to run fetches
this slowly on some platforms...

-- brion

Brion Vibber

2011-09-26 21:43:09 UTC