gvSIG CE at FOSSGIS 2013

Dear all,

FossGIS 2013 is taking place from 12th - 14th of June `13  in Rapperswil, Switzerland.
All information regarding registration, location and program of the event can be found at [1].

I would like to announce some topics regarding gvSIG CE and invite all of you to join:

- "Geoprozessierung mit Open-Source DesktopGIS": Benjamin Ducke will present gvSIG CE with SEXTANTE and GRASS GIS, which together provide a powerful tool for all matters relating to the theme "geoprocessing":
http://www.fossgis.de/konferenz/2013/programm/events/574.de.html

- "Kartenerstellung und Bearbeiten von Legenden mit gvSIG CE": In this 1.5 hours workshop, Ruth Schönbuchner will give an overview on gvSIG CE capabilites regarding labelling, legend generation and map layout production:
http://www.fossgis.de/konferenz/2013/programm/events/575.de.html

- Share interests and discussion at gvSIG CE BoF (Birds-of-a-Feather)-meeting on thursday, 13th of June `13 room no. 1.271 (table 1):
https://www.fossgis.de/wiki/2013/Sprints_und_BOFs

We are looking forward to meet you at FossGIS in about 2 weeks!
Best regards,
Ruth Schönbuchner

[1] http://www.fossgis.de/konferenz/2013/

Evaluation of how to implement a cartesian coordinate system support (EPSG:5806)

Hi all,
some of the gvSIG CE users need this CRS, so we are evaluating how to implement it.
Please, if you can, read the wiki, add anything you consider important and share your thoughts.
Regards,
Victor.

4 developers more with write access to our SVN

Dear all,
if you haven't followed the gvSIG CE developer list, maybe you don't know that we have now 4 new developers with write access to our SVN. All of them are well known FOSSGIS developers:

- Victor Gonzalez is a Computer Engineer by the Valencia Politechnical University. Victor was awarded with the 52º North Student Innovation Prize in 2009 with the project “SQL Script Profile for 52°North WPS-T”. He has participated in projects like GearScape, GGL, or gvSIG. He is making an amazing work in gvSIG CE since more than one year developing new functionalities, hunting bugs, working on the integration of GeoTools in gvSIG CE, getting the code base into better shape, etc.:
- Export map layouts to raster formats
- Geotools integration methodology
- clean-up work

- Jorge Arévalo is a Computer Engineer by Universidad Autónoma de Madrid, UAM. He has been collaborating with PostGIS and GDAL creating the PostGIS Raster GDAL driver. He has been solving some GDAL related open cases. gvSIG CE has now an amazingly good GDAL drivers support.

- Micho Garcia studied Ocean Sciences at the Vigo University, and has devoted his professional live to GIS development. He has been updating the EPSG database

- Francisco Puga is an ICT engineer by formal training and has participated in several NGOs related to technology, development and free software. He has sent us some patches for the java version of SEXTANTE:
- Models are not loaded by means of the "load button" in the settings
- GridCalculator does not work with models in batch mode
- Error in sextante.history file prevent use the toolbox

We really appreciate the work you are doing.
See you soon
Jose Canalejo

gvSIG CE using GDAL 1.9.2 to read raster data, including ECW, JP2, MrSID and Lidar files

Recently, I've been working for gvSIG CE project, thanks to the folks of geomati.co (I'm part of the team now!). I was solving some GDAL related open cases (this, this and this). the aim is to get a functional and stable gvSIG CE 1.0 version. There are still a couple of cases to solve (this and this), but I think we're pretty close. I'd like to explain what I did to solve these cases.

Make gvSIG CE work with GDAL 1.9.x

This bug was hard to find but easy to fix.  First, a 1-minute introduction to GDAL use in gvSIG CE. gvSIG CE uses a JNI wrapper (based on GDAL 1.7.x GDAL packed JNI wrapper) to make C++ native calls to GDAL library. There are 2 important GDAL JNI wrapper classes involved in most GDAL operations:

Gdal.java (it interacts with native C++ GDALDataset class)

GdalRasterBand,java (it interacts with native C++ GDALRasterBand class)

Both classes inherit from the same base class: JNIBase. This class defines a function named baseSimpleFunctions that calls one of several native C functions, depending on an input parameter. But some of these native C functions are GDALDataset-based, and other are GDALRasterBand-based. I mean, the pointer passed to them doesn't hold the same C structure (wrapping C++ object) in all cases.

And here was the thing. To actually read the raster data, the Java function GdalRasterBand::readRaster is called. This function checks the position and size of the region being accessed, comparing it with the raster dimensions. To check this, the C functions getRasterXSize and getRasterYSize are called. These functions expect a pointer to a GDALDataset C object, but the GdalRasterBand Java class provides a pointer to a GDALRasterBand C object. I had to replace those Dataset-based calls to RasterBand based calls, to match the underlying C structure.

Replace ECW and MrSID driver with GDAL

First, a few comments on these formats. ECW stands for enhanced compression wavelet. This propietary format is a high-performance image-compression format designed specifically for geospatial imagery. It's patented and owned by Intergraph. More information here.

Intergraph provides a license needed SDK for encoding data and a free SDK for reading. Since version 4.1, this free SDK is only available for Desktop development on Windows systems. There are promises to include Linux support since 5.0 version, but gvSIG CE doesn't plan to support ECW and other proprietary formats. So, I worked with the last known version working on Linux systems too. This is 3.3.

About MrSID, it stands for multiresolution seamless image database. It's a proprietary format too, owned by LizardTech. As happens with ECW, there's a free SDK to read MrSID files. It can be downloaded from here (MrSID SDK, under Tools & Utilities. Free register required). And as happens with ECW too, gvSIG CE won't directly support this format. So, it delegates in GDAL.

In both cases, we'll need to compile GDAL with the proper support. So, I had to download and compile MrSID and ECW libraries and build GDAL using --with-ecw and --with-mrsid options.

As there is a JNI wrapper for GDAL, there are JNI wrappers for ECW and MrSID. The goal was: get rid of them, and give GDAL credentials to handle these formats. Four steps involved here:

- Compile GDAL with ECW and MrSID support, as said

- Tell GDAL to take care of those formats for reading (class GdalDriver)

- Unregister ECW and MrSID drivers from gvSIG CE raster library (libRaster)

- Delete ECW and MrSID wrappers and native code.

Once done, you have gvSIG CE using GDAL 1.9.2 to read raster data, including ECW, JP2, MrSID and Lidar files (previously managed by ECW and MrSID drivers).

Jorge Arevalo

SEXTANTE news and inofficial preview

Dear All,
As some of you may already be aware, there have been substantial changes to the SEXTANTE project.
The website at www.sextantegis.com now focuses on a new implementation of SEXTANTE, written in Python and designed primarily to run from within QGIS.
In response to this, we have contacted Victor Olaya, the original creator of SEXTANTE, and he agreed for us to transfer the Java source code to the gvSIG CE SVN. From now on, we will keep developing SEXTANTE as an official part of the gvSIG CE project.

Over the past few weeks, I have made extensive changes to the SEXTANTE settings system and the graphical Modeler. You can download a snapshot of the current SEXTANTE version here
Copy the ZIP into "../extensiones/es.unex.sextante" and unpack it, overwriting any existing files. You might also want to delete your old settings in the "Sextante" folder in your home directory.

The most important improvements are:
1. Renovated settings pages.
2. Better portability: GRASS, SAGA and R are by default looked for in subfolders of the SEXTANTE installation folder.
3. Brought back the ability to set up a GRASS installation anywhere on your system as an algorithm provider.
4. Vastly improved graphical modeler. This tool is now stable and should be fit for productive work.
5. GRASS vector modules work on 64 bit Windows.

For now, you will also need to select "High compatibility mode" on the GRASS settings page. This will make sure that SEXTANTE runs with any version of GRASS 6.4.x.
I would especially like your feedback on the new graphical modeler (note that the documentation is currently out of date). If you find any problems, please report them via the bug tracker (don't forget to select "SEXTANTE" as current project first).

Best,
Benjamin Ducke
gvSIG CE Team

Running gvtools from sources

After some work we are able to share some results, basically a methodology [1] that can be "described" and followed by more than one person. We have created a new empty repository where user interface functionality is added little by little from the old codebase and the old non-GUI calls are replaced to calls to a new, in-progress Geotools-based library.

Some of these functionalities are very easy to include, some require changes to the GT-based library and take more time. In any case, now we can estimate roughly the cost of the integration since we have been tracking the time spent and we know how many functionalities are still to migrate. Anyway, we'll see how many people is interested on it enough to give us a hand and share the effort.

We have created a project on github that contains the work in progress [2]. As it can be seen in the developer guide [3], a "git clone" and a "mvn package" is enough to make it ready to run. You can find some general description of the project here [4].

Best regards!

[1] http://gvsigce.blogspot.de/2012/11/geotools-integration-methodology.html
[2] https://github.com/gvtools/gvtools
[3] https://github.com/gvtools/gvtools/blob/master/doc/source/introduction.rst
[4] https://github.com/gvtools/gvtools/blob/master/doc/source/developer_guide.rst

Geotools integration methodology

The Munich codesprint was some time ago and we did not have the opportunity to devote more time to the Geotools integration since then. Finally we have been able to work on it for a week.
As stated from the start, the aim of the 2012 codesprint in Munich was not to finish the Geotools integration but to get an idea of how to tackle the task and estimate the cost of it (see our Release Planning for details on how we want to tie this into the CE release cycle).
First of all, for all that have not dealt with the gvSIG code base before: We have found that this task is not easy at all. The gvSIG 1.x branch (the one CE is based on) has hundreds of thousands of lines of code, organized in ~50 different projects that not only do not completely separate user interface from model but that are also tightly coupled with each other. Little or no documentation, duplicated code, meaningless variable names and 17.000 warnings makes handling the gvSIG source code a really discouraging experience. If you're a developer, I'm sure you get the idea. 
After trying different approaches and restarting from scratch each time, we have finally found a method that has the following advantages:

• It provides value immediately. We'll be able to produce gvSIG versions working on Geotools very soon. Of course, the initial versions will lack a lot of functionality the current gvSIG CE version has, but:

1. everybody will be able to see the progress of the integration
2. at a certain point in time, some people may be content with the status of the integration and start using it

• It defines several more or less precise steps to follow. This is helpful when dealing with a beast of so many lines of code. The method is explained further down.

• Can be done in parallel by several people. This is very important since this initiative has received some attention and I would say it is likely that someone will join and share the efforts. The development will take place in a GIT repository in github, so that forks and merges can happen easily. 

This method has been the first one we have considered good enough to make the integration succeed. Of course, suggestions are welcome.
Let's explain the method. In gvSIG, there is a plugin system, called "andami" that hosts about 25 extensions that provide the user interface. These extensions use several libraries to implement their functionalities. Instead of modifying the existing code base, we have created one more extension project called "main" and one more library project called "core".
The main extension will receive one by one every functionality from the old codebase. As the libraries used by the old code are not present in main, there will be some compilation errors. To solve these compilation errors, the code that uses the old libraries will be changed to use the core library, adding the necessary functionality to core if necessary. Of course, core exposes Geotools API, as it was the main aim of the integration. 
So schematically:

• Move all extensions to main one by one

• For each extension:

• Include all classes in the same extension that are necessary to compile. Code can be removed if:

• Supports old 0.3 version (method name ends with 03)

• It is not necessary right now and we'll be forced to add it later. For example:

• Public methods that are not called in main -> Either they will be called with code that will be moved to main and we'll recover the erased method, or they will never be called so it was well removed.

• Whole classes -> At the end we can compare old and new codebase and we can detect the classes that are missing.

• Fix the compilation errors of code using old libraries (not accessible anymore from main). This might require adapting the code to the new core library and even adding more functionality to it. In case of new functionality:

• Discuss in the list the best way to implement

• Implement

• Add javadoc

• Add unit tests 

Some considerations when following the method:

• Our first priority is integrating Geotools. Please don't try to fix the extension code in main. While following the previous method you will see some things that can easily be fixed. Please don't. You'll realize that there are many of these and fixing them will keep you from the real aim, which is the integration. Let's try to do only minimal changes to the code that goes to main.

• Do not add TODOs. They are ignored. Assert false:"todo message"; is much more effective. If you don't want to implement a method, try to remove it (following the rules stated before).

• If you add some comment related to the integration, include the text "gtintegration" in it, so that we can process it later.

• Please, try to understand core before making changes. Just use the mailing list to agree on some changes before starting to code.