An example of reporting with ArcGIS Pro, ReportLab and ArcPy.

The discovery of ArcGIS Pro

Before going to the Esri User Conference in San Diego in July 2017, I must admit that I never really had focused on ArcGIS Pro capabilities. As a geographer trained on GIS since 2003, and having worked in operational frameworks, on complex topics, shame on me some will say.

The Esri demo, every time, make you dream about what you could do, even what you could have done in your past projects. Hundreds of ideas and visualizations overwhelm you: interactivity, 3D, decision support, visualization support etc. Nevertheless, before being able to use ArcGIS Pro, I had to wait a couple of months. Last September an incredible project arrived. We had to help a client to compare ArcMap and ArcGIS Pro functionalities and tools to help them migrate to ArcGIS Pro soon.

Some will say that the new interface is user friendly, more intuitive than ArcMap. I started my training on ArcView 8.3 and I followed the evolution of the software until ArcMap 10.5 (and even 10.6 now). To be honest, it took me quite some time to get used to ArcGIS Pro and to find the functions and tools I use every day. However, after a week or so, I could appreciate how fast and powerful ArcGIS Pro is. Task manager, open different maps and different views in the same project, 3D indoor and outdoor are incredible tools to discover (or re-discover), play with. But you need time.

And of course, not everybody has the time, whether to test ArcGIS Pro capabilities, or to learn and find the alternatives when a tool is not yet available. The interface is nice to use and mechanisms come back very quickly, but what happens when local, regional or federal agencies wish to migrate entirely from ArcMap to ArcGIS Pro? For having tested it, migrating from ArcMap to ArcGIS Pro is not a question of being user-friendly anymore, but rather a question of accessibility and availability.

The context: a survey of a possible migration from ArcMap to ArcGIS Pro

A couple of months ago, one of our clients requested to study a possible migration from its tools, native or custom made, from ArcMap to ArcGIS Pro. A total of forty tools have been analyzed and tested in their current work environment, considering the future needs (obsolete tools, tools not working with ArcMap 10.5 etc.). Next these tools and functions have been compared to the tools and functionalities available in ArcGIS Pro 2.0. Finally, we ran tests with ArcGIS Pro to validate or not some propositions we made after the first analysis. After a month and a half of analysis, our results showed that:

  • It would be better to wait for the next ArcGIS Pro version, which will be more complete and which will answer some expectations (some native tools were not yet available at the time of the survey),
  • Custom-made tools must be redeveloped to be compatible with the ArcGIS Pro SDK environment.

Of course, some tools could be used as it is in ArcGIS Pro, but this would force the client to navigate between two environments, which presents some advantages (for example users would be trained and operational when the migration to ArcGIS Pro is total), and some drawbacks (training time for people working in the field or in emergency contexts is always time consuming, not everyone is open to changes, this represents an investment in terms of computers).

As we could not talk about 40 tools and as most of them are confidential anyway, we chose a tool that a large amount of people use with ArcMap but cannot use as a native tool in ArcGIS Pro yet. We are talking about reporting. This tool was particularly interesting because in the ArcGIS Pro environment you need to be willing to dive into python and ArcPy. In the next chapters, we are going to describe how to use the python package ReportLab to make easy reports and tables.

How to use ReportLab within ArcGIS Pro?

I can already hear a lot of people, developers and users of python within an ArcGIS framework say: “Nothing new here!”. Indeed, if you spend time online on the forums, blogs, anywhere you can find information and documentation on this package. However, as I said earlier, not everybody has the time to look, nor the time to become a developer, or just spend time learning from scratch. For a beginner with python and ArcPy, or even for a regular ArcMap user, used to using report wizard and report designer, using ReportLab is like entering a new world. First, you must:

  • Dive into the use of this package (although installing it is very easy),
  • Become familiar with python grammar,
  • Learn how to use the python console in ArcGIS Pro (not everyone uses it not even in ArcMap),
  • Learn how to use or re-discover ArcPy.

To summarize, you must be patient, you need time not only to test the package and the libraries, but also to try to understand the python structure. You also need to take advantage of forums where you are most likely going to find answers to your questions. Much more than in the official documentation anyway, as it can be somewhat lacking in information. If you have these qualities, then you are ready to start! And with a couple of hours, days, spent on this package, including reading forums and asking a lot of questions, the result is worth it and ReportLab is a very good tool.

1.     How to install ReportLab and use arcPy SearchCursor to extract data from an attribute table and display it in a pdf report?

Our client uses automatic reports in ArcMap to inform a partner agency about updates made in a particular dataset. Once the report is received (in a table format), the agency can check data and validate or not the updates. In this post we will see how to:

  • Install ReportLab in ArcGIS Pro
  • Use the platypus library
  • Use ArcPy in ReportLab (within the python console in ArcGIS Pro)

a)     Install ReportLab in ArcGIS Pro

Click on the Project Tab (ArcGIS Pro must be open). Your screen should look like this:

Then click on Python in the side panel. Click on Add packages. Search for ReportLab and install it.

Once the install is finished, you can go back to your map and test if the package has been properly installed. You can use as a tutorial the official ReportLab documentation and a blog that I particularly recommend, Mouse vs. Python. You can now create a simple PDF by entering the following code in your python console:

The PDF is generated in the folder where your current ArcGIS Pro project is saved under the title “hello.pdf”.

In our example, what we want to achieve is to edit a table in which we will display some fields of an attribute table we chose. On top of this this, we need to filter the data. Living in Geneva, I chose to download the open data from the SITG. I will use the building GDB.

Imagine that the Meyrin city council requested a survey of all the buildings on the city territory, with the level count, the height of the buildings, and specific identifiers. I start by defining a definition ensemble. Once this is done, I will create my report using ReportLab, the platypus library and the ArcPy SearchCursor.

b)     How to use platypus and the SearchCursor?

Platypus will allow you to generate tables. You call it in your code as you did for importing the canvas earlier.

from reportlab.platypus import Table, TableStyle

To filter the fields in your attribute table you will need to use a SearchCursor. First you need to import ArcPy :

import arcpy

After importing ArcPy you need to give your workspace, the fields you want to display, and then the SearchCursor.

Here I put the full code, without all the tests and failures I ran into. Before arriving there I had to learn and understand what tuples are and how to use them. Even here, the result is clearly not satisfactory: the table is too big, there is no title, no date and the data are not filtered. However, we are close to what the client is expecting.

How to make this table and the query better? We need to work on the style of the table. The code already contains style elements:

from reportlab.lib.pagesizes import A3, A4, portrait, landscape

from reportlab.platypus import SimpleDocTemplate, Table, TableStyle, Paragraph, Frame, Spacer

from reportlab.lib.units import cm

from reportlab.lib.styles import ParagraphStyle, getSampleStyleSheet

from reportlab.lib.enums import TA_LEFT, TA_RIGHT, TA_CENTER, TA_JUSTIFY

from reportlab.lib import colors

These lines import style libraries:

  • Page size and orientation
  • Document, table and paragraph,
  • Units (here in centimeters)
  • Margins
  • Colors

These lines need to be updated. For example, the colors you want to use in the titles of your columns:

table.setStyle(TableStyle([

(‘INNERGRID’, (0,0), (1,1), 0.25, colors.black),

(‘BOX’, (0,0), (1,1), 0.25, colors.black),

(‘BACKGROUND’,(0,0),(4,0),colors.lightgrey)]))

Same for the style of the entire table :

table = Table(data, [6 * cm, 6 * cm], repeatRows=1)

Here again, I recommend you reading and spending time on the blog Mouse vs. Python for good explanations and on StackExchange Geographic Information systems for good advice and even examples of code you can try and adapt to your needs.

2.     How to improve the style of your table?

The biggest part of the work was to make the table and extract the right fields from my attribute table. However, the time spent on the “aesthetics” of the table and the report must not be neglected. We will show you how to add:

  • A title
  • Date and time of the report

At the beginning of the code I imported: import datetime

In my code, I added the following lines:

mydate = datetime.datetime.today()

#Header Titre du rapport avec date et heure

header = Paragraph(‘Récapitulatif des bâtiments et de leurs hauteurs. Commune de Meyrin. {}’.format(mydate.strftime(‘%c’)) * 1, styles[‘title’])

w, h = header.wrap(doc.width, doc.topMargin)

header.drawOn(canvas, doc.leftMargin, doc.height + doc.topMargin h)

I also adapted the size of my columns function of the lenght of my text:

table = Table(data, colWidths=[2*cm, 3*cm, 3*cm, 5*cm, 5*cm], repeatRows=1)

Then in my SearchCursor I added a SQL clause to filter my data according to the level count:

data = [tuple(headers)]for f in arcpy.da.SearchCursor(fc, fields, sql_clause=(None, ‘ORDER BY NIVEAUX_HORSOL ASC’)):

data.append((x for x in f))

I wanted the title of the report on every page:

#Construire le rapport en répétant le titre sur toutes les pages

doc.build(elements, onFirstPage=_header, onLaterPages=_header)

Here is my entire code:

My report is very long as it contains all the buildings of a city. However, the result, even looking simple, allows me to have a template using ReportLab and ArcPy within my ArcGIS Pro environment. I can reuse this code every time I need to extract, filter data and make a report. If you are waiting for the report wizard and report designer, this is an excellent alternative.

The model proposed here can be easily adapted. We made other tests especially with multiple tables on a page, changing the font size, colors, adding a logo etc.

With a logo the code has been modified like this :

from reportlab.platypus import SimpleDocTemplate, Table, TableStyle, Paragraph, Frame, Spacer, Image

elements=[]

logo = r’C:\Users\Melanie\Documents\ArcGIS\Projects\Test_evaluation\ITISGIS_verysmall.png’

im = Image(logo, 3*cm, 3*cm)

elements.append(im)

Conclusion

Aside from tables, we tried to generate pdf forms, calculus, stats, graphs etc. Once the difficulty of diving into python and ArcPy (which I can easily understand being a beginner), ReportLab is a flexible, complete and useful tool.

References

http://www.reportlab.com/docs/reportlab-userguide.pdf
https://www.blog.pythonlibrary.org/2010/03/08/a-simple-step-by-step-reportlab-tutorial/
https://www.blog.pythonlibrary.org/2010/09/21/reportlab-tables-creating-tables-in-pdfs-with-python/
https://www.blog.pythonlibrary.org/2012/06/27/reportlab-mixing-fixed-content-and-flowables/
https://opendata.swiss/fr/organization/sitg-systeme-dinformation-du-territoire-a-geneve
https://pro.arcgis.com/fr/pro-app/arcpy/functions/searchcursor.htm
https://www.programiz.com/python-programming/tuple