Excel

1. Excel reporting
1.1. Introduction
Considerable effort is expended by most organizations in the production of management
information, reports that provide the information to support decisions.
While there are a number of automated solutions available, the vast majority is still prepared
manually using Excel. Not only is this time consuming, but there is a significant risk of error
due to the manual nature of the process. Security in Excel is limited (and rarely applied) and
there is no audit trail to identify who made changes.
With an automated solution, the time previously used for report preparation can be used for
more valuable activities such as analysis and interpretation of the information and decision
support.
1.2. What you will learn
In this article we will cover the following topics:
 How to connect Excel to a database using ActiveX Data Objects (ADO).
 How to build a report using Visual Basic for Applications (VBA) base on data in a database.
 Techniques to optimize the performance of your reporting solution.
1.3. What you need to know
The following is assumed:
 A basic understanding of VBA for Excel.
 A basic understanding of databases and SQL.
1.4. Excel uses
Excel is it own worst enemy. It has such rich functionality that it is often used for purposes for
which there is a much better solution. Extreme versions of this that I have seen include
flowcharting using the drawing functionality or a presentation slide pack. These are obviously
areas that Excel functionality was never intended to satisfy. However, there is a whole raft of
areas that use core Excel functionality that I believe are wrong.
Let’s examine some of these:
 Transaction processing. I often see core business processes performed in Excel. It may
be that the Accounts payable or receivable ledger is maintained in Excel.
 Accounting adjustment creation. Provisions, accruals and allocations are often calculated
in Excel and then either loaded to the accounting system or exist as part of reporting
adjustments outside the accounting system.
 Budgeting. There is a lack of decent budgeting / forecasting solutions at a price acceptable
to the majority of organisations on the market and Excel is almost always used to perform
this function.
 Reporting. In most organisations Excel will form part of the reporting suite. While it is OK
to use Excel for reporting it is the manner of creation that I object to. Normally a data
download is taken from the core system and the report created using Excel formulae like
Vlookup and Sumif.
So, what is wrong with this? I believe there are a number of generic issues with Excel:

 Multi user. While you can share a workbook, Excel does not really do this well – it is not a
database management system that can deal with record locks. Generally, each
subsequent save overwrites previous saves for those cells that have been edited.
 Audit trail. Excel has no audit trail, you can delete lines / columns and edit cells with no
record of who did it or when it was done, of course, you do not need to log-in to Excel like
you do for most database applications. Therefore, it is impossible to tell who entered /
removed / edited data.
 Dimensionality. Excel can handle 3 dimensions as a result of the nature of worksheets and
workbooks which are organised into sheets, rows and columns. While pivot tables /
PowerPivot allow the analysis of multi dimensional data, I frequently see Excel attempting
to store multidimensional scenarios.
So what should Excel be used for?
1.5. Reporting using Excel
As I have mentioned, I believe that Excel is a perfectly good reporting tool as long as all
transactional and adjustment data is stored in a proper systems that manages areas such as
audit trail and referential integrity.
Excel can connect to databases in a number of ways. Originally we had Microsoft Query to
connect to the database, execute the reporting logic via a SQL statement and present the
resulting data in Excel. Form Excel 2007 onwards, we could do that directly. We can use plain
tables to present transaction data or pivot tables to analyse the data. Where the in-built
functionality starts to be found lacking is the production of standard reports which often have
complex formatting that standard Excel functionality cannot automatically perform.
This is where we need to consider developing our own functionality. There are a number of
different ways to achieve the connectivity to databases but I have found Microsoft’s Activex
Data Objects (ADO) able to meet all my needs. This library has a number of very nice
functions / properties that have persuaded me to stick with it.
There are two major objects in ADO which you will need; the connection and recordset objects.
I will discuss these later in this article.
1.6. ADO basics
So how do we use ADO in Excel? As with all libraries we can either reference the library and
create objects using the New key word or we can create the object using the CreateObject
funcion.
1.6.1. Referencing ADO
ADO can be referenced as follows:
 Launch the VBA editor (I use Alt+f11 to do this).
 Insert a new module into the current workbook.
 Choose Tools, Reference from the menu. The following dialogue is shown:

 Scroll down to find the ADO library. You will find several versions of this, I generally use
version 2.7 or 2.8.
 You can then start entering code as follows:
Function Open_Connection()
Dim con as ADODB.Connection
Set con=new ADODB.Connection
1.6.2. Using CreateObject
The CreateObject function allows you to not reference the ADO class library but, if you use this
approach, the VBA editor will not be able to prompt for available methods / properties of the
objects or enumerations.
The code for this is as follows:
Function Open_Connection()
Dim con as Object

Set con= CreateObject(“ADODB.Connection”)
1.6.3. The connection object
The connection object represents a connection to a database. Once you have an open
connection to the database you can do anything to the database; add/ edit / delete data,
retrieve data, and modify the database schema as long as your user account has the rights to
do so.
ADO can connect to a number of different databases (Microsoft SQL Server, Oracle, MySQL
etc), either directly through ADO drivers for the database or through ODBC. The connection is
made by defining the connection parameters or connection string and using the Open method.
The connection string is specific for each database type. You do not need to remember the
syntax for connection strings as there are several sites on the web for this, I recommend
www.connectionstrings.com.
Here is example code to open a connection to a Microsoft SQL Server database:
Function Open_Connection() as ADODB.Connetion
Dim con as ADODB.Connetion
Set con = new ADODB.Connetion
con.Open “Driver={SQL Server}; Server=[Server_Name];
Database=[Dataabse_Name]; UID=[User_Name];PWD=[Password];”
Set Open_Connection = con
End function
Where:
 [Server_Name] is the name or IP address of the SQL Server instance.
 [Database] is the name of the database to connect to.
 [User_Name] and [Password] are the user credentials.
Of course you can pass the user name and password to this function from a user dialogue.
1.6.4. Error trapping
ADO can generate errors, for example, as the result of invalid login credentials or a syntax
error in a SQL statement. If these are left un-trapped the user will get nasty VBA error
messages. In order to prevent this we can use the On Error statement to prevent these errors
appearing to the user but we probably need to take account of the error in our program.
The connection object has two properties that help here:
 The State property returns a boolean indicating whether the connection is open (logged in)
or not and can be used to test whether a login attempt was successful.
 Errors.is a collection of items that contain details of errors returned by the Connection
object and can be used to return the description of the error.
The following code shows the Open_Connection function modified to return a message if the
log-in is not successful:
Function Open_Connection() as ADODB.Connetion
Dim con as ADODB.Connetion
Set con = new ADODB.Connetion
On Error Resume Next
con.Open _“Driver={SQL
Server};_Server=[Server_Name];_Database=[Dataabse_Name];_UID=[User_Name];PWD=[Pa
ssword];”

If con.State = 0 then
MsgBox "There was a problem logging in. " & _
con.Errors(0).Description, vbCritical, "Log in"
End If
Set Open_Connection = con
End function
1.6.5. The Recordset object
The Recordset object represents the results of a SQL select statement and is used to retrieve
data. The Recordset object contains a number of useful methods and properties.
1.6.5.1. Open method
This opens the recordset. The recordset can be a table, a view, a SQL select statement or (for
those databases that support it) a stored procedure that returns records. The following
example function returns a recordset:
Function Get_Data (con as ADODB.Connection) as ADODB.Recordset
Dim rst as ADODB.Recordset
Set rst=new ADODB.Recordset
rst.Open “Select * from tblData”, con
Set Get_Data = rst
End Function
The Open method also accepts a number of optional parameters:
 CursorType can be used to identify the type of cursor, the default is **.
 LockType defines the record locking. The default is ** which allows you to edit data in the
recordset. I prefer to use Recordsets only to retrieve data and so always use the setting
adLockReadOnly.
 Options. This is used to instruct the ADO driver the type of SQL statement being passed,
e.g. the name of a table, a view or a SQL statement. Most of the time, I find that this is not
required, the ADO driver / database engine works this out. However, with Microsoft
Access, I find I need to specify the type of statement or an error ocurrs.
1.6.5.2. Retrieving the data
The Recordset will contain rows and columns. The columns can be referenced using the
Fields property. Fields is a collection representing the columns in the recordset. The value of
a particular column in the current row is referenced as follows:
rst.fields.item(0).value
Because Fields, Item and value are the default properties, this can be written rst(0). The Fields
collection also supports the Count property to determine the number of columns in the
collection. The collection is zero based so the ordinals range from 0 to Count-1.
It is also possible to determine the name of the column using the Name property, e.g.
rst(0).Name and the data type using the Type property, e.g. rst(0).Type.
Traversing the rows can be done with a number of self-explanatory methods, MoveFirst,
MoveLast, MoveNext and MovePrevious.
We can tell when we get to the end (last row) of the recordset using the EOF property. This is
a Boolean which returns True when the cursor is beyond the last record.
The following routine writes the contents of a recordset to an Excel sheet:

Sub Write_To_Sheet(rst as ADODB.Recordset)
Dim intRow as Integer, IntColumn as Integer
intRow=1
Do Until rst.EOF
For intCols = 1 to rst.Fields.Count
ActiveSheet.cells(intRow, intColumn)=rst(intColumn-1)
Next
rst.MoveNext
intRow = intRow+ 1
Loop
End Sub
In order to include the column names in the first row, the following may be inserted:
ActiveSheet.cells(1, intColumn)=rst(intColumn-1).Name
Next
The EOF can also be used to test whether a SQL statement returns any rows at all.
1.6.6. Closing the recordset
Once you have finished with the data from the recordset you should always close it using the
Close method to release system resources and to prevent run-time errors if you attempt to
open the recordset again while it already open.
1.6.7. State property
The Recordset object also has a Boolean State property which can be used to test whether the
recordset opened successfully.
1.7. Report building
1.7.1. Report structure
In my opinion, reports have the following elements:
 Title – describes what the report is showing and any data filters applied (e.g. date range,
products, region, business division, etc.)
 Criteria – any user applied filters on the report data.
 Headers – these are the row and column headers or dimensions of the report and describe
the contents of each row or column (e.g. cost captions (nominal accounts), months,
products etc.
 Body – this contains the numerical data.
 Calculations – values calculated from the data in the body, may be simply row or column
totals or more complex.
The following image shows these elements.

1.7.2. Report generation
How do we generate such a report programmatically with VBA?
Let’s suppose that we are constructing a profit and loss report from an accounting database.
1.7.2.1. Database schema
For reporting, most databases are designed having a ‘star schema’. This means that there is a
single data table containing the numerical (value) data and foreign keys to each of the master
data tables (e.g. nominal accounts, products, etc.) where descriptions of the dimension
members are held.
Data
Data
FK1 Foreign key 1
FK2 Foreign key 2
FK3 Foreign key 3
FK4 Foreign key 4
FK5 Foreign key 5
Master data 1
PK Primary key
Description
Master data 2
PK Primary key
Description
Master data 3
PK Primary key
Description
Master data 4
PK Primary key
Description
Master data 5
PK Primary key
Description
The data table should not, if any degree of normalization has been performed, contain the
descriptions of the dimensions (master data) that is represented but the index or code value
from the master data table.
1.7.2.2. Named ranges
Named ranges are a feature in Excel that is underused. It allows the user to define and
reference a range of cells using a name. I use this extensively when generating reports.
When a block of data is retrieved from the database, I assign a name to the range of cells that
the data has been inserted into. You can then use the named range for a variety of purposes:
 Determine the number of rows / columns.
 Use in cell formulae.

 Apply formatting.
1.7.2.3. Row and column headers
Row and column headers can be inserted into the worksheet using the technique shown in
section 1.6.5.2.
In the example, the row headers are the nominal accounts from a table called Nom_AC. The
table has 4 columns that we are interested in:
 idAc – the primary key which is an auto-incrementing integer. This is the field that is
referenced from the data table.
 Code_Ac – this is the nominal account code used by the accounting function.
 Description – the decription of the account.
 Type – the type of account, e.g. profit and loss (Value=1), balance sheet (Value=2).
The report needs the first three of these fields on the sheet so the SQL statement is “Select
idAc, Code_Ac, Description from Nom_Ac where Type=1”
Let us also assume that we have a style called Row_Header that we wil apply to the resulting
data block and that we will apply a name to the range that the data is inserted into.
The code to insert the row header is as follows:
Sub Row_Header(con as ADODB.Connection,intStartRow as Integer)
Dim intRow as Integer, IntColumn as Integer
Set rst = new ADODB.Recordset
rst.Open “Select idAc, Code_Ac, Description from Nom_Ac where Type=1”, con
intRow=0
Do Until rst.EOF
ActiveSheet.cells(intRow+intStartRow, intColumn)=rst(intColumn-1)
Next
rst.MoveNext
intRow = intRow+ 1
Loop
rst.Close
Set rst = Nothing
Names.Add “Nominal_Accounts”, Range(cells(intStartRow, 1), Cells(intRow +
intStartRow-1, 3))
Range(“Nominal_Accounts”).Style=”Row_Header”
End sub
The first column contains data we do not need to display to the user so it can be hidden.
1.7.2.4. Data blocks
The report now has row and column headers and we now need to insert the data into the
range of cells that are the intersect of the headers.
In order to insert the data there are a number of approaches. The simplest is to generate a
SQL statement for each cell in the range and insert the value returned into the cell. The code
is as follows:
Sub Data(con as ADODB.Connection)
Dim intRow as Integer, intColumn as Integer

For intRow = 1 to Range(“Nominal_Accounts”).Rows.Count
For intColumn = 1 to Range(“Months”).Columns.Count
rst.Open “Select Sum(Val) from Data where idAc=” &
Range(“Nominal_Accounts”).Cells(intRow,1) & ” and idMonth=” &
Range(“Months”).Cells(1,intColumn)
ActiveSheet.Cells(Range(“Nominal_Accounts”).Cells(intRow,1),Row,
Range(“Months”).Cells(1,intColumn).Column) = rst(0)
rst.Close
Next
Next
Set rst = Nothing
End Sub
We can, of course, apply a range name and formatting to the block we have inserted.
1.8. Performance optimization
1.8.1. Minimising the database interactions
Communicating with the database can be slow, especially if the database contains large
amounts of data, is not properly indexed or the SQL we are executing is complex. Of course,
each time we execute a SQL statement the database engine will take an amount of time to
process the request. If the database is on a database server, there may also be network
considerations. Therefore, it is good practise to minimize the communication with the
database.
In the example in section 1.7.2.4, a SQL statement was executed for every cell in the report.
Let’s assume we had 200 nominal accounts and 12 months on our Profit and Loss report. This
means we executed 2,400 SQL statements!
Additionally, there may have been no data for some of the cells. The SQL statement would
have returned a zero for these but we have potentially wasted computing resource here and
therefore compromised performance.
A faster way is to retrieve a recordset that contains the data we want and then work out where
to insert the data into the report using VBA.
In order to do this the SQL statement will be:
Select Sum(Val) as Val, idAc, idMonth
from Data
Group by idAc, idMonth
In order to retrieve only data that is relevant to the profit and loss, we should also insert a filter
to our SQL based on the Type column of the Nom_Ac table. The SQL is as follows:
Select Sum(Val) as Val, idAc, idMonth
from Data inner join Nom_Ac on Nom_Ac.idAc=Data.idAcc
Where Nom_Ac.Type=1
Group by idAc, idMonth
This will return a recordset containing a number of rows with each row containing a column for
the value, the nominal account id and the month id. We now need to work out where to put
each value. This is done by looking up the Nominal account id and month id in the row and
column headers.
We could write VBA to perform the lookup on the sheet but it is quicker to use the inbuilt Excel
functions (e.g. MATCH) to do this. We can use Excel worksheet functions from VBA as
follows:

Application.WorksheetFunction.Match()
When using the Match function there are a couple of things that we need to be aware of:
1.8.1.1. Single dimension lookup range
We need to supply a 1 dimensional array (i.e. one row or column) as the lookup range
parameter, the Row headers are 2 dimensional ranges and we need the 1st row or column of
them. This can be done using the Rows or Columns property of the Range object:
Range(“Nominal_Accounts”).Columns(1)
1.8.1.2. Data types
The data types of the value being sort must match those in the lookup range, i.e. if your lookup
range contains numerical data, make sure that you supply a numerical value as value to be
matched.
So, in order to get the match, the following code might be used:
Dim intRow as Integer, intColumn as Integer
rst.Open “Select Sum(Val) as Val, idAc, idMonth
Where Nom_Ac.Type=1
Group by idAc, idMonth”, con
Do until rst.EOF
‘Find the row for the data
intRow = Application.WorksheetFunction.Match(rst(“idAc”,
Range(“Nominal_Accounts”).Columns, 0)
intRow = intRow + Range(“Nominal_Accounts”).Row -1
‘Find the column for the data
intColumn = Application.WorksheetFunction.Match(rst(“idMonth”,
Range(“Months”).Rows, 0)
intColumn = intColumn + Range(“Months”).Row -1
ActiveSheet.Cells(intRow , intColumn) = rst(Val)
rst.MoveNext
Loop
rst.Close
Set rst = Nothing
End Sub

1.8.2. Minimising interaction between Excel and VBA
Another thing that slows VBA down is interaction with the host application, in this case, Excel.
For example, reading or writing data from or to a cell takes time.
It is little known that you can exchange data between a range of cells and a VBA array with a
single command rather than having to traverse the range and array. VBA arrays are held in
memory and so can be accessed and manipulated very quickly. Therefore, if you can hold the
data in a VBA array and only read or write to the worksheet when you absolutely have to, then
performance will improve.
Two additional facts can help us speed up our code. The first is the GetRows method of the
ADODB.Recordset object. This allows you to copy the entire contents of a recordset either to
a range of cells in a worksheet or to an array variable in a single command:
varData=rst.GetRows()
The resulting 2 dimensional array has the recordset columns as dimension 1 and the rows as
dimension 2. This means that effectively the reocrdset has been transposed by the GetRows
method.
The second is that the Application.WorksheetFunction methods (e.g. Match, Transpose) can
use an array variable rather than a range of cells.
Using these two facts we can dramatically reduce the number of interactions between the VBA
code and Excel. We can insert the row headers as follows:
Sub Row_Header(con as ADODB.Connection,intStartRow as Integer)
Dim varData() as variant
rst.Open “Select idAc, Code_Ac, Description from Nom_Ac where Type=1”, con
‘Populate the array variable from the recordset and transpose at the same time
varData=Application.WorksheetFunction.Transpose(rst.GetRows())
rst.Close
Set rst = Nothing
Names.Add “Nominal_Accounts”, Range(cells(intStartRow, 1), Cells( intStartRow +
Ubound(varData,1), 3))
‘Place the array onto the sheet
Range(“Nominal_Accounts”) = varData
Range(“Nominal_Accounts”).Style =”Row_Header”
End sub
Using this approach the data is transferred to the worksheet in a single operation rather than a
cell at a time. In our example of 200 nominal accounts this would require accessing 600 cells
(3 columns for each nominal account) so we have reduced 600 operations to a few operations.
In order to insert the data into the intersects, we can also use an array variable. But to
minimise the number of interactions with the Excel sheets when looking up against the row and
column headers, we should copy these to array variables. We will also need to create an array
variable to represent the range of cells that we are going to insert the data into.
Dim intRow as Integer, intColumn as Integer, intData as Integer
Dim varData() as Variant, varRows() As Variant, varColumns() As Variant, varInsert()
as variant
rst.Open “Select Sum(Val) as Val, idAc, idMonth

Where Nom_Ac.Type=1
Group by idAc, idMonth”, con
varData = rst.GetRows()
rst.Close
Set rst = Nothing
varRows=range(“Nominal_Accounts”).Columns(1)
varColumns=range(“Months”).Rows(1)
Redim varInsert(Ubound(VarRows), Ubound(varColumns))
For intData = Lbound(VarData) to Ubound(varData)
intRow =Application.WorksheetFunction.Match(rst(1),varRows,0)
intColumn =Application.WorksheetFunction.Match(rst(2),varColumns,0)
varInsert(intRow-1, intColumn-1) = rst(0)
Next
With ActiveSheet
Range(.cells(Range(“Nominal_Accounts”).Row, Range(“Months”).Column),
.Cells(Range(“Nominal_Accounts”).Row+ Range(“Nominal_Accounts”).Rows.Count -1,
Range(“Months”).Column + Range(“Months”).Columns.Count -1)) = varInsert
End with
End Sub
1.9. Conclusion
We have explored how to generate reports in Excel from source databases and how to
optimize the code for performance. The techniques covered were all taken from our product
DBReport.
DBReport is an in-Excel Report writer that allows the creation of reports that can be refreshed
at the touch of a button and can explot the rich formatting functionality of Excel.
For more information please visit our website: www.pendragonsystems.com.
1.10. About the author
Jason Raikes is a Chartered Accountant. For the past 20 years he has worked as a
management consultant specialising in helping clients implement management information
solutions. Jason is also the developer of a number of software products focusing on the
automation of management information and marketed by Pendragon Systems Limited.

Excel

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (18)

Similar to Excel

Similar to Excel (20)

Excel