The FRB and SDMX:National data and InternationalstandardsSan CannonFederal Reserve BoardSDMXConference9-11 January 2007
SDMX Conference9-11 January 2007 2Background• The Fed is a statistical agency as well as acentral bank and regulatory agency.• Lots of data and information are available on thepublic website.• Statistical data are varied: monthly industrialproduction indexes, daily interest and exchangerates, quarterly financial flows for various sectorsof the economy, surveys of small businessesand consumers, etc.
SDMX Conference9-11 January 2007 3Serving our users betterTo some, it may appear that the statistical agencyrole is secondary.• Data are not always easy to find.• Downloads are not customizable.• Example: Trying to extract one industrialproduction series requires retrieving two textfiles, cutting and pasting, reformatting….• Complete – yes. User Friendly – no.
SDMX Conference9-11 January 2007 4Data Download Program (DDP)• XML designated as key format but project teamwrestled with implementation details.• Staff weighed a homegrown DTD setup againstthe new SDMX standard.• SDMX looked to have greater benefits and wasadopted.• Good decision: additional internal applicationsas well as interagency projects using SDMX arein the works.
SDMX Conference9-11 January 2007 5A lot to learn• SDMX is based on data structure definitions(‘key families’) and codelists, with every conceptrepresented by a code with a correspondingdefinition.• We were unfamiliar with this type of datamodeling, so it proved challenging.• Two of our pilot datasets translated easily to thisnew format; others needed more work.
SDMX Conference9-11 January 2007 6Data structures differSome data structures are readily adapted to theconcept/codelist representation. Series “keys”have no real mnemonic value.HBBA Int. Rate, Official, Discount rate/Base rateHBCA Int. Rate, Official, Intra-day loansSCBA Indust. Production, Motorvehicles, NSASCBB Indust. Production, Motorvehicles, SA
SDMX Conference9-11 January 2007 7Hierarchical relationshipsWe allow data to be modeled hierarchically, and usemnemonics that reflect this.RIFSPFF_N.BR.*:RateR.I.*:Rate of interest in moneyand capital marketsR.I.F.*:Federal Reserve SystemR.I.F.S.*:Short-term or moneymarketR.I.F.S.P.*:Private securitiesR.I.F.S.P.FF.:Federal funds_N.:Not seasonally adjusted.B:Business (Five days, Monday-Friday)JQI_I02Y3361T3_N.M:J.*:Indices except of pricesJ.Q.*:ProductionJ.Q.I.:Industrial_I.*:NAICS-based industryclassification02Y:codes from year 20023361.:Motor Vehicle ManufacturingT:thru3363:Motor Vehicle PartsManufacturing_N.:Not seasonally adjusted.M:Monthly
SDMX Conference9-11 January 2007 8Applying the SDMXmodel• Data represented by a concrete number of concepts aremuch easier to represent with key family dimensions andattributes:JQI_I02YMF_N.M → Topic_Industry_SA.FreqFA156900005.Q Prefix (2 digits), Sector(2),→instrument type (5), series type (1), frequency• Hierarchical relationships and varying number of conceptsmakes life more difficult:RIFSPPNA2P2D30_N.B → Topic?_SA.FreqRIFLGFCY20_XII_N.B T→ opic?_Inflate_SA.Freq
SDMX Conference9-11 January 2007 9Decisions we made• Allow a variable number of data structuredefinitions per dataset.• Use the compact format for internalexchange and external downloads.• Stick with SDMX 1.0, for now.• Use a relational database to store dataand XML information for retrieval.
SDMX Conference9-11 January 2007 10Final product: DDP!• We have a flexible application whose interfaceis entirely driven by the data structuredefinitions.• “We store the XML as carefully sliced text in arelational database and we can build an indexstructure that allows us to respond to ad-hocqueries very efficiently, even for large volumesof data.”Data Download Program
SDMX Conference9-11 January 2007 11Strengths and weaknesses• Because interface is entirely data driven, it iseasy to add new data.• Internal architecture is complex, due to securityand the data workflow:– SDMX files are generated by data staff andtransmitted to public website staff for processing.– These files are made available on the website and“shredded” for database entry and lookup.• Current structure is not set up for codelistsharing.
SDMX Conference9-11 January 2007 12What do the users say?• “Really excellent. One of thebest I’ve used on the web.”• “This Data Download thing isbetter than sliced bread.”• “I downloaded the XML files, but Icannot run them. All I see is thexml code.”
SDMX Conference9-11 January 2007 13What do the numbers say?• More than 250,000 unique visits since April –about 50,000 per month.• Data Download Program is the 6th most visitedarea on the Federal Reserve website.
SDMX Conference9-11 January 2007 14Next steps• Add more data: reserves, exchange rates,consumer credit.• Continue working with other central banks andstatistical agencies on common framework.• Prepare to move to SDMX 2.0 to take advantageof additional features.
SDMX Conference9-11 January 2007 15The last slide…Questions? Comments?Thank you for your attention!San Cannonscannon@frb.gov(202) 452-3710