California County Information Technology Projects


Published on

This report accompanied the Placer County Implementation of AVATAR presentation that review Placer\'s lessons learned throughout the process.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

California County Information Technology Projects

  1. 1. California County Information Technology Projects Statewide Information Sharing Overview, Status, and Lessons Learned A Mid-Process Review of Placer County’s Behavioral Health System Implementation of Netsmart’s AVATAR Products: California Practice Management (Cal-PM 2005) Management of Services Organization (MSO 3.1) Clinicians Work Station (CWS 2004) Prepared and Presented by: Placer County Health & Human Services Management Information Systems March 12, 2007Placer County – Health & Human Services – Management Information Systems 1 of 31
  2. 2. This Page Intentionally Left BlankPlacer County – Health & Human Services – Management Information Systems 2 of 31
  3. 3. Table of ContentsExecutive Summary.........................................................................................................................4Introduction......................................................................................................................................4Project History and Original Project Scope.....................................................................................4AVATAR Cal-PM.........................................................................................................................12 AVATAR Cal-PM Overview....................................................................................................12 Placer’s Scope of AVATAR Cal-PM Implementation..............................................................14 AVATAR Cal-PM ChallengesIssues in Implementation ........................................................17AVATAR MSO.............................................................................................................................19 AVATAR MSO Overview........................................................................................................19 Placer’s Scope of AVATAR MSO Implementation..................................................................21 MSO ChallengesIssues in Implementation..............................................................................22AVATAR CWS.............................................................................................................................24 AVATAR CWS Overview........................................................................................................24 Placer’s Scope of AVATAR CWS Implementation..................................................................26 AVATAR CWS ChallengesIssues in Implementation.............................................................26Overall Implementation Considerations........................................................................................27 Buy vs. Build.............................................................................................................................27 National and California Based AVATAR User Groups............................................................28 Vendor Participation..................................................................................................................28What Have We Learned From This Adventure?...........................................................................29Contact Information.......................................................................................................................31 Placer County – Health & Human Services – Management Information Systems 3 of 31
  4. 4. Executive SummaryThis report presents an annual system review conducted by the Placer County’s Health andHumans Services, Management Information Systems (HHS-MIS) Department over the life of theManaged Care Software Project from its inception in 1999 through its current day status as ofMarch 1, 2007. These annual reviews originally defined and subsequently redefined the scope ofthe Managed Care Software Project resulting in updates to the Strategic Plan for HHS-MIS tomeet the evolving requirements of the California Department of Health Services (DHS),Department of Mental Health (DMH), and Department of Alcohol and Drug Programs (ADP);consistent with changing state and federal reporting requirements.IntroductionThe goal of this system review document is to share with our counterparts throughout the state ofCalifornia, an overview of Placer County’s Managed Care Software Project, the current status,and most importantly, lessons learned throughout the implementation of AVATAR to meet ourBehavioral Health - System of Care information system requirements. To accomplish this, wehave presented a history of the project with key project milestones, followed by the dynamicscope of our multi-year project. We have also addressed our partnerships with our vendor(Netsmart), AVATAR California User Groups, county consortiums, and peer informationsharing amongst counties and other Netsmart customers across the country.Project History and Original Project ScopeThe Placer County Department of Health & Human Services was established in 1995, with agoal of better serving our clients and the community at-large through integrated services,universal intake, and centralized administration.This new County Department merged what was then the Department of Health & MedicalServices, the Welfare Department, and other social services functions into one mega-department,Health and Human Services. Comprised of nearly 1/3 the County’s workforce, we now haveover 1,100 employees and have a total operating budget of $140 million. 1999 - Project InitiationIn 1999, the Warner Group conducted a comprehensive review of the IT needs within HHS, as asupplement to the County’s Information Technology Master Plan. The highest businessapplication need identified by this effort was for a Behavioral Health Managed Care softwaresystem. Placer County – Health & Human Services – Management Information Systems 4 of 31
  5. 5. 2000 – Project Scope DefinitionThis Behavioral Health Managed Care Software system would address the needs of a number ofservices that were integrated into 2 main Systems of Care -- Adult and Children’s.These services are listed below: Adult System of Care Children’s System of Care• Mental Health Services • Child Protective Services• Mental Health Billing • Child Welfare Services• Drug & Alcohol Services • Foster Care Placement• Drug Court • Crisis Services (Adult & Child)• In-Home Supportive Services • Juvenile Drug Court• Public Guardian • Children’s Receiving Home• Adult Protective Services• Older Adult Services• Vocational Rehabilitation• Assisted LivingThe list gives you an example of how we have tried to integrate services, and the challenge wehave to select a software system that will adequately address the data and financial trackingneeds of such a variety of services and functions. Placer County – Health & Human Services – Management Information Systems 5 of 31
  6. 6. Additional Services Needing New Information SystemsThe Warner Group Study also envisioned that the same software application could eventually beexpanded to include functionality for a number of other, PHYSICAL HEALTH services andprograms, most notably: • Fee-For-Service Primary Health Managed Care Program, • Community Clinics, including: •MIA (Medically Indigent Adult outpatient clinic) •Women’s Health •Dental •Pharmacy • Children’s Medical Services • Community Health & Public Health ProgramsThese programs are currently using a legacy system, rapidly approaching its shelf life. We werehoping our new application would have the capacity and “hooks” to include these programs. What Would a New Software Application Do?We envisioned a new software application to have a number of concurrent capacities to enable usto be a more robust user of behavioral health client and service information.Most notably, we were looking for a new application to: • Permit the collection, storage & tracking of client information, (including outcomes); • Replace an existing, antiquated mental health billing system with a more comprehensive, behavioral health practice management information system; and • Provide management with much needed information regarding client services and how they are delivered. 2000 – Needs AssessmentIn the Spring of 2000, we began an internal needs assessment to determine the specific featuresthat would be essential for a new software application. This step took most of the Summer of2000 to complete, so by early Fall, we were ready to develop the Feasibility Study Report (FSR).We formed focus groups comprised of key HHS staff who could be potential users of the newinformation system. Placer County – Health & Human Services – Management Information Systems 6 of 31
  7. 7. We organized the focus groups by system function, according to the following categories: • Direct service clinicians; • Service authorizations, claims review, payment processing • Client appointment scheduling/Registration/Demographics • Client Eligibility • Financial/Accounts Receivable • ManagementJust to cover all the bases, we tried to make sure that we had a good integrated mix of staff from: • Behavioral Health Services, • Physical Health Services, • Financial and Support Services, • IT, and • ManagementThis effort involved about 70 HHS staff, and served as an excellent “sounding board” for thefolks who will ultimately wind up using the system on a day-to-day basis. We also received a lotof valuable input regarding the “look and feel” of a new system, in addition to its functionality.Documenting Data Requirements & Business ProcessesConcurrent with the Focus Group discussions, IT Project staff performed a detailed analysis ofbehavioral health; addressing the business requirements from various aspects. This included: • Examining the interface needs of our legacy systems; • Conducting 135 staff/user interviews; • Reviewing report & form requirements; and • Mapping existing business processes.The kind of analytical steps listed above were essential when we prepared the Feasibility StudyReport (FSR), the Request for Proposal (RFP), and made our selection process much easier,because we had already done much of our homework.It actually helped to have two relatively new IT Analysts assigned to this project. They wereable to ask good questions about how and why things work, and became well-grounded in ourbusiness processes very quickly.We were hoping that the business process mapping experience would also facilitate theapplication implementation process, and help us explain how we do business to our potentialsoftware vendor. Placer County – Health & Human Services – Management Information Systems 7 of 31
  8. 8. What To Do With The Needs Assessment Information?By Fall of 2000, we were ready to take all the information and suggestions we had collected overthe summer, and boil them all down to a Feasibility Study Report (FSR).Our Project Team consisted of a good cross-section of the stakeholder groups: • Behavioral Health Services, • Physical Health Services, • Financial and Support Services, • IT, and • ManagementWe began by looking at all the features the focus groups indicated that were either absolutenecessities or just “nice-to-haves”Each feature was sorted by functional area, and then checked to make sure we had no duplicates.These features were also clarified as much as possible. 2001 - Functional RequirementsTo make sure we had described each feature correctly, we then returned the list to the focusgroup participants.There were a few items that were either clarified or consolidated from this step.We ended up with a list of 185 items. Each one was reviewed for its criticality, and assigned agrade, with a higher weight assigned for those functional requirements that were absolutelymandatory. 2001 - Feasibility Study Report (FSR)The FSR is our County’s internal document to ensure that each software request addresses all theimportant questions that might come up when selecting a vendor. It also is a way to satisfyCounty IT and the CEO’s Office that we have done our homework and that we can justify theeffort.Placer County has a standardized approach to all FSR’s. County IT has developed guidelinesand a mandatory format to follow, to ensure that all technical questions are addressed in the sameway, in the same sequence. Placer County – Health & Human Services – Management Information Systems 8 of 31
  9. 9. In addition to the list of functional requirements, the FSR included the technical specifications,hardware requirements, and how the new information system will work with our CountyNetwork and other interface needs. 2001 - Request for Proposals (RFP)From the FSR, it was relatively easy to develop the RFP.In addition to placing the overall project into perspective, the RFP gave us the opportunity to tellour story to prospective vendors, giving them an idea of the journey we have traveled to get tothis point, and what we expected of them as traveling companions.Of course, the list of 185 functional requirements served as the heart of the RFP, and gave thevendors some concrete standards to respond to.We worked closely with our County Procurement Services Division to fashion a document thatadhered to all legal and ethical guidelines for fairness, openness, and consistency. We made surethat we included clear guidance regarding how proposals must be submitted, how they would beevaluated, the project responsibilities for both the vendor and the County, and formats for costsummaries and itemized breakdowns, both for one-time and ongoing costs.Included in the RFP was a series of appendices that gave vendors some added understanding ofthe current County environment and what lengths to which we had already gone in documentingour needs: Appendices: • Acronym List • County Network Diagram • County Network Access & Security Standards • Business Process Flowcharts • System Functional Requirements • List of Required State & County Reports • Standard County Software AgreementImportantly, we included a copy of our standard software contract, as a starting point for vendordiscussions.Getting the Word OutDuring our lengthy discovery period, we had developed a fairly extensive list of potentialvendors. Each of these 60 firms was sent a copy of the RFP, in either hard copy or CD format.In some cases, the firms requested both.We also placed the announcement on the County’s Web Site, for broader audience outreach. Placer County – Health & Human Services – Management Information Systems 9 of 31
  10. 10. Our timeframe included 3 key dates: • RFP Release: September 10, 2001 • Bidders’ Conference: October 16, 2001 • Proposal Submittal Deadline: November 13, 2001What Happened?Out of the 60 firms contacted, six responded with proposals.Phase 1 – Determination of ResponsivenessOf the six firms submitting, only 2 complied with the requirements laid out in the RFP. TheResponsiveness Screening was performed by our Procurement Services Division.Phase 2 – Functional EvaluationOf these two finalists, both scored a passing grade of 80% on our mandatory requirements list.The Functional Evaluation was conducted by our HHS MIS Division through a pre-developedscoring process.Phase 3 – Evaluation of Responsive ProposalsFor the final two proposals, we assembled a 13-member evaluation team comprised of keystakeholders in each of the functional areas. Also included was a representative from the CEO’sOffice, to provide added perspective.Once we had all read the proposals, we invited each vendor to provide a day-long demonstrationof their product.We then assigned a small group of Evaluation Team members to conduct reference checks overthe phone. Evaluation Team members contacted their counterparts in each California Countyreferenced, getting perspectives from: • Program staff • Fiscal and accounting staff • IT staffWe then reconvened as an entire Team, and shared the results of the reference checks.Finally, each individual Team member scored each vendor on the prescribed criteria provided inthe RFP. Placer County – Health & Human Services – Management Information Systems 10 of 31
  11. 11. 2002 - Vendor SelectionProcurement Services scored and totaled the results. Early in February 2002, we made ourselection. The Board of Supervisors approved our request to enter into negotiations with theselected vendor Creative Socio-Medics (now Netsmart Technologies) on February 26th, 2002.The three modules of AVATAR selected to meet Placer’s requirements were the CaliforniaPractice Management (Cal-PM) module with two additional modules consisting of ManagedServices Organization (MSO), and Clinicians Workstation (CWS). 2002-2003 - Pre Go-Live ActivitiesOver the course of 15 months, Placer County worked with the vendor to develop a plan forimplementation of the AVATAR products to meet the functional requirements of the contract.This involved a variety of activities to include software & hardware delivery, training,configuration, testing, more training, more configuration, and more in depth testing. To meet atight implementation schedule, the AVATAR Project Team worked diligently to meet a June 1st,2003 implementation date successfully. 2003 - AVATAR Go-Live ActivitiesAVATAR Cal-PM went live on June 1, 2003. With this milestone reached, data was convertedfrom the existing Yolo Billing system allowing for a single consolidated master client record forBehavioral Health clients that could be accessed by all modules of the AVATAR product. Thiswould include Cal-PM, MSO, and eventually CWS, allowing us to realize the immediate benefitof a master patient record shared by multiple systems that would eventually feed into our HHSData Warehouse, SMART, providing accurate statistics of the population we serve.In the following sections, we will address the current status of implementation for the threemodules we are implementing. In doing so, we will provide an overview of the functionality ofthe module and then a description of the scope of implementation for that module followed bychallenges encountered through this phase of implementation. Placer County – Health & Human Services – Management Information Systems 11 of 31
  12. 12. AVATAR Cal-PMTo address Placer’s specific implementation of AVATAR Cal-PM, it is helpful to understand thebasic functionality of the AVATAR Cal-PM Module. Below you will find a brief description ofAVATAR Cal-PM as provided by Netsmart Technologies in the section titled, AVATAR Cal-PM Overview. Additional, information on Netsmart’s products can be found at the overview of the AVATAR Cal-PM module are sections addressing Placer’sspecific implementation of this module as well as challenges and issues faced during theimplementation.AVATAR Cal-PM OverviewNetsmart’s AVATAR California Practice Management (Cal-PM) is the result of a coordinatedeffort between Netsmart and several California Counties to meet the dynamic state requirementsof county Mental Health Plans (MHP) and Alcohol and Other Drug (AOD) programs specific toCalifornia Department of Mental Health (DMH) and California Department of Alcohol and DrugPrograms (ADP). Examples of these requirements include Short-Doyle/Medi-Cal claiming, CSIReporting, MHSA Reporting, Drug Medi-Cal claiming, and Cal-OMS reporting.The foundation behind Netsmart’s AVATAR Cal-PM module is the national AVATAR PracticeManagement (PM) module which is the result of nearly 30 years of experience in the humanservices industry, combined with today’s software development technology. AVATAR is acomprehensive behavioral healthcare software solution that provides a common gateway to asuite of robust software modules that are equipped to handle everything from the newestdevelopments in SQL reporting, automated Methadone dispensing and outcome analysis, to thetraditional workhorses of billing, tracking, scheduling and treatment planning in all types ofbehavioral healthcare modalities.Core functionality of Cal-PM includes the following functional areas:Client ManagementClient admission, account management, leave, discharge, treatment and census data are recordedon various screens within this module. This module includes a collection of reports that are vitalto the internal business procedures of our clients: Current Unit Census, Bed Availability Reportand tables of active and discharged patients. A historic record is created for each episode of care,containing all service, diagnosis, treatment and billing information. Bed management, dischargeplanning and a number of administrative functions are supported within this module.California Required EDIAs part of the Client Management module, the California Required EDI menu options containthe functionality to meet California specific reporting requirements. Enhanced reportingfunctionality has been developed to meet the recently modified CSI/DIG reporting requirements,Cal-OMS reporting requirements, MHSA (FSP, KET, QTR) tracking requirements, and OSHPD Placer County – Health & Human Services – Management Information Systems 12 of 31
  13. 13. reporting requirements. Functionality is continually updated to meet the changing requirementsof Medi-Cal, DMH, and ADP.PractitionerStaff records are maintained in this module. This is also used as a key cross reference forensuring that only eligible providers can render particular services under user defined benefitplans.Billing/Billing ReportsThis module manages both the billing and accounts receivable functions of the system. It recordsall subscriber and plan coverage information, including charge input, bill preparation, paymentposting, client ledger, and electronic claims submission and remittance. Major billing formats(including HIPAA transaction sets) are supported “off the shelf." This eliminates a need forthird-party billing services. This module provides management and accounting reports to controlthe patient revenue/managed care contracts. The reports available include Aged AccountsReport, Report of Charges, Payment/Adjustment Report, Daily Transaction Report, ActiveReceivables and Detail Trial Balance.Scheduling/Scheduling ReportsThe AVATAR Cal-PM scheduling module is tightly integrated with the systems billingcomponents. Patients are scheduled and as the appointments are kept and services are rendered,all relevant billing information is sent to the patient’s ledger. Service recording can easily bedistributed to the provider level. Several pre-defined client, staff and facility reports derived fromscheduling data are available in this module.RADPlus Utilities and System Maintenance (AVATAR System Administrator Utilities)This module provides three system maintenance areas: • Client Maintenance provides the tools to delete erroneous clients, program movements and service entries. Tools are also provided to change a client’s medical records number and purge the call intake file. • System Definition provides the tool to build the default system tables for Guarantors, Benefit Plans, Service Codes, Hospital Bed Files, Referral Sources, Payment and Adjustment Codes, CPT Codes and Revenue Codes. These tables are used to determine billing rates and guarantor coverage, and to maintain accurate client treatment histories. • RADPlus Utilities provide Netsmart clients with control of the system security and configuration (i.e. menus), as well as full control of content management. Placer County – Health & Human Services – Management Information Systems 13 of 31
  14. 14. Placer’s Scope of AVATAR Cal-PM ImplementationOver the course of the next three years from June 2003 through May 2006, AVATAR Cal-PMwas implemented across Behavioral Health’s System of Care. Below is a diagram of the highlevel AVATAR Process Flow that is used as a roadmap in implementing AVATAR in newbusiness areas to track clients, services, programs, and outcomes. A summary of ourimplementation is shown in the following sections on the next several pages. AVATAR Process Flow CWS 2004 Cal-PM 2005 MSO 3.1 Clinician’s Work Station Practice Management Management of Services Organization Start Client Search Master Patient Index Admission Initial CSI /MHSA/Cal-OMS/Other Req. Data Collection Financial Eligibility Care Authorization Assessments /Progress Notes/ Appointment Scheduling Claim Adjudication Treatment Planning/Other Chart Entry Client Charge Input / Workflow Management Review Post Staff Activity Log / Provider Payment Processing MSO Charge Posting Diagnosis State Required Reporting CSI/MHSA/Cal-OMS and Other Reporting 837 HIPAA Claiming 835 HIPAA Remittance Processing / Payment Posting Client Discharge Cost Report End Diagram: Placer County AVATAR Process Flow Placer County – Health & Human Services – Management Information Systems 14 of 31
  15. 15. Placer County Health and Human Services currently uses AVATAR Cal-PM as its primaryPractice Management information system for Behavioral Health. Because Placer Countyoperates under the Health and Human Services integrated agency model, Placer provides acomplex array of services through a variety of programs under the umbrella of BehavioralHealth; tracking client and service information for 13,144 unique clients last fiscal year.Additionally, Placer tracks a number of client families for clinical and financial eligibility. Inconjunction with tracking clients by program participation, we have designated an equallycomplex array of Guarantors which offer us the ability to claim services rendered out to a varietyof funding sources discussed in more detail in the Guarantor Setup section of this document.Program SetupTo support the complex requirements of our Behavioral Health Systems of Care, Placer hasdeveloped 194 programs to meet a variety of state tracking, reporting, and claimingrequirements. To summarize our program types, we currently have Programs defined for thefollowing program areas: • Central ACCESS Intake • ACCESS Emergency Response • Mental Health O/P, Day Tx, Residential • MHSA - Mental Health Services Act • Alcohol and Drug Programs • Adult Protective Services • In Home Support Services • Cal-Works • Child Welfare Services • Families First • Family Connections • Family Preservation • R.A.F.T. – Rallying Around Families Together • 26.5 Individual Education Plan • Managed Care • Fee For Services • Organizational Contract Provider – Sierra Family ServicesEach of these programs are unique and come with their own reporting requirements bothinternally to the county and externally to the state as well as various fiscal intermediaries such asMedicare and Medi-Cal. It is important to point out that for our business practices; we havedefined a variety of programs to meet a number of Placer County System of Care requirements,and not only to meet our Medi-Cal (Medicaid) claiming requirements. Placer County – Health & Human Services – Management Information Systems 15 of 31
  16. 16. Guarantor/Payor SetupSimilar to our Program setup, Placer has an equally complex array of Guarantors/Payors that aredefined in a client’s Financial Eligibility to designate what services are paid for and by whichGuarantors. We currently have 55 active Guarantors which can be grouped in the followingcategories: • Medicare • Medi-Cal • Grants • Special Funding (MHSA, SB90) • Private Insurance • Self-PayFamily Pay • UMDAP • General Fund AccountsPractitioner SetupCurrently, Placer has approximately 1,026 registered Practitioners in AVATAR Cal-PM whohave provided services to approximately 13,114 unique clients last fiscal year. Of those 13,114active clients, 1,454 clients were referred out for behavioral health services in FY 2005-2006(soon to be tracked in AVATAR MSO). Of these practitioners, many of them provide servicesto clients admitted in multiple programs both independent of each other and concurrently. Wehave a large number of practitioners who provide services for Mental Health, Alcohol and Drug,and Child Welfare programs. With the ability to hold multiple licenses and provide services toclients in multiple programs, each of our Registered Practitioners require extensive setup in ourCal-PM system to eventually claim services out to any number of fiscal intermediaries.Service Code SetupPlacer has opted to use a mix of standard CPT codes and custom service codes to describe,classify and report both billable and non-billable services performed by practitioners. Asampling of these service codes is shown by the following categories of Mental Health, Alcoholand Other Drug, Miscellaneous Services, and Non-Billable Services.Approved MH CPT Codes: • (90801) Assessment • (90802) Interactive Assessment • (90804) Individual Therapy - 20 to 30 min • (90862) Medication Support - M.D. Only • …..Approved ADP CPT Codes: • (H0004DF) ODF-Individual Counseling, PPWP • (H0015DA9) Day Care Rehab (DCR), SACPA, PPWP, CHILD • (H0019DF9) Perinatal Residential (RES), LT, SACPA • (H0020DG9) NTP-Methadone, SACPA, PPWP • ….. Placer County – Health & Human Services – Management Information Systems 16 of 31
  17. 17. Approved G-Line Payment Codes: • (GL-0807) Daycare • (GL-0801) Education • (GL-0826) Enrichment Activities CalWorks • (GL-0812) UA Testing • ….. AVATAR User Base Currently, Placer has approximately 315 registered and active AVATAR end users that span several business areas across Health and Human Services (HHS). To put the number of users into perspective, Placer County HHS has approximately 1191 employees. Of these 1191 HHS employees, 592 employees work for Adult and Children System of Care, making the percentage of users at approximately 53% to date. This number will near 100% upon full implementation of AVATAR MSO and CWS when almost all System of Care staff will be required to do some portion of their work in AVATAR. Currently, our end users are represented by staff from the following functional areas: Clerical, Administrative Support, Accounting, Case Managers, Social Workers, Program Supervisors, Program Managers, and Directors. AVATAR Cal-PM ChallengesIssues in Implementation Challenge #1 – Project Staffing As the number of end users have increased over the past three and a half years since Placer went live with AVATAR, MIS Staffing has had to increase as well to continue to move the project implementation forward and keep up with end user demand, changing state requirements, and planned implementation activities for AVATAR MSO and CWS. AVATAR MIS Project Staffing 200 200 200 200 200 200 200RoleCaledar Year 7 6 5 4 3 2 1 2000Dedicated MIS Resources Supervisor (Louie/Larry) 0.25 0.25 0.5 1 1 .75 .25 Project Manager (Eric/Jules) 1 1 0.75 .5 1 1 Implementation Analysts (Pete H./Lonnie) 1 1 1 1 1 1 1 1 Implementation Analysts (Kathy) 1 0.5 Implementation Analysts (Elaine) 0.5 Report Writer Help Desk Analyst (Bal) 1 1 1 1 Info. Technology Techs (Pete K./Candice) 1 0.5 1 1 1 Administrative Technicians (Diana) 0.5Indirect MIS Support Network Administrator (Lonnie) 0.1 0.1 0.1 0.1 0.1 0.1 Database Administrator (Larry) 0.1 0.1 0.1 0.1 0.1 0.1 Client PCHelp Desk Admin. (Bob K./John P.) 0.1 0.1 0.1 0.1 0.1 0.1Total Resource Allocation 6.05 5.05 4.55 4.3 3.3 2.55 2.25 2 Placer County – Health & Human Services – Management Information Systems 17 of 31
  18. 18. A number of alternatives to minimizing the challengesissues faced with project staffing arecommon and frequently used by agencies across the state. These alternatives include: increasingpermanent staff levels to meet the long term goals of the organization; outsource projectmanagement tasks to qualified and experienced professionals; contract with software vendor forprofessional services to assist in the implementation of a new system. Each of these alternativeshave their pros and cons.For Placer, the decision was made to gradually increase MIS staffing over the course of theproject to move the implementation forward as new business areas were brought online withAVATAR and to meet the business requirements of the growing user base. In addition tointernal resources, professional project management services from the vendor were included inthe contract to assist in the overall implementation of AVATAR.The down side of this strategy was that there were not sufficient MIS resources allocated to theproject to meet the original requirements and scope of the project. As noted above in the MISProject Staffing table, the position of project manager remained vacant from March 2003 throughApril 2005 causing unexpected delays in the overall implementation of AVATAR Cal-PM.Similarly, the vendor rotated multiple project managers through our project to work on theimplementation of specific functionality such as MH Medi-Cal claiming, CSI, Drug Medi-Calclaiming, and appointment scheduling. At the time, the turn over and subsequent increases instaffing affected the timeliness of our implementation due to the extended learning curverequired to get up to speed with not only Behavioral Health business processes but themethodology used for meeting those business process requirements through the implementationof AVATAR.Challenge #2 – Project ScopeThe system was originally acquired to support 6 major business areas Mental Health Programs,Alcohol and Other Drug Programs, Community Health, Community Clinics, Pharmacy andDental Clinic. A Cyclical implementation was planned to implement four of the productsmodules (PM, MSO EMR, and a 3rd party Pharmacy). In 2005, management re-defined the scopeof the project to only include AVATAR implementation in Behavioral Health.Challenge #3 – Paradigm ShiftThe implementation approach presented a paradigm shift to the organization. Business processeswere reengineered to place the system at the front of the processes versus at the end.Challenge #4 – External Factors (HIPAA)There were significant changes occurring at Federal and State levels that impacted timelines ofdeliverables. HIPAA has required a great deal of resources from both program and MIS areas toensure that all requirements are being met by their associated deadlines.Challenge #5 – Product Readiness (2002-2003)The product did not meet all of the functional requirements out of the box; many capabilitieswere under consideration if not under development by the vendor. Over the course of ourimplementation, multiple enhancements were delivered to meet site specific requirements of ourcontract. Placer County – Health & Human Services – Management Information Systems 18 of 31
  19. 19. AVATAR MSOTo address Placer’s specific implementation of AVATAR MSO, it is helpful to understand thebasic functionality of the AVATAR MSO module. Below you will find a brief description ofAVATAR MSO as provided by Netsmart Technologies in the section titled, AVATAR Cal-PMOverview. Additional, information on Netsmart’s products can be found at the overview of the AVATAR MSO module are sections addressing Placer’s specificimplementation of this module as well as challenges and issues faced during the implementation.AVATAR MSO OverviewAVATAR Managed Services Organization (MSO) software for Case Management is a systemwhich serves the specialized needs of states, counties, Managed Care Organizations, specialtynetworks, and providers attempting to carefully monitor both at-risk and non-risk contracts. Keyfeatures include: • Contract tracking (patients, providers and other) • Service request management • Authorization tracking • Case management documentation • Capitation revenue management • Costs by CPT codes, physician, patient or period • Claims adjudication and payment calculations based on negotiated fee schedules • Multifaceted contract tracking • Integration with electronic claims • Payment and/or GL/AP systems • EnrollmentMember status and eligibility verification are entered into the system to begin the enrollment andservice process. Eligibility for one or more Payor sources are verified and entered into the systemeither via automated loads or manual data entry.ScreeningMember requests/needs are assessed and evaluated during the screening process to determine theappropriate level of care. The user has the ability to create their own assessments using theRADplus administrators tool set.Service AuthorizationAuthorizations are linked to Benefits or Contracted Services in the Contract function.Authorizations are referenced for claims adjudication/review, calculated in accumulators forIBNR, and reporting on utilization and profitability. • Records and tracks authorizations for member services Placer County – Health & Human Services – Management Information Systems 19 of 31
  20. 20. • Automatically assigns authorization number • Selects benefit through link between Member Plan Assignment • Authorizations and contracts with specific plansCase ManagementThis function allows the user to perform case management on an individual member to determinecontinued level of care required, document notes, and monitor review dates. Authorizationextensions are also performed in conjunction with transitions of level of care.Claims ProcessingClaims data is received from the provider via electronic file, log data, or paper claims, and areinput into the system for validation and adjudication of service data against authorization data. Apayment recommendation is made by the system for review and approval. Once a batch has beencreated, a file is generated to be sent to a GL/AP system for checks production and paymentprocessing. Explanations of Benefits (EOBs) are also generated to accompany the payment to theprovider.Reports • Claims processing inquiry • Claim appeal inquiry • Authorization inquiry • Concurrent review inquiry • Provider-authorized dollars • Provider IBNR claims • Claims paid by provider • MSO average cost per service per member • MSO average cost per member per month • Case manager authorized dollars • Generate hard copy authorizations • Provider appealed authorizations • Current authorizations by provider • Generate GL/AP report • Claims paid within 30 daysMaintenanceThrough the maintenance functions, the user has the ability to set up contract requirements, plandefinitions, service codes, fee schedules and provider credentialing. Other functions include,loading of the MEDS MMEF eligibility file, employee registration, member merge andAuthorization Approve/Pend/Denial Rule definitions. Placer County – Health & Human Services – Management Information Systems 20 of 31
  21. 21. Placer’s Scope of AVATAR MSO ImplementationOver the past 11 months, Placer County has undergone extensive testing through a variety ofconfigurations of AVATAR MSO.In the end, we have determined that we have opted for a higher level configuration of MSO atthe level where the System of Care and Physical Health branch out under HHS. Through thisdetailed trial and error period of testing multiple configurations, we have settled on the oneconfiguration that best meets our business practices and that configuration is summarized below.Funding Source SetupA single funding source for System of Care is defined to track authorizations for all BehavioralHealth and Non-Behavioral Health Services tracked within the System of Care. Supplemental tothe System of Care Funding Source Definition, Placer intends to utilize the Budget TrackingSolution to track Service Authorizations by Budget Account to accomplish the required trackingof fiscal accounts. • System of Care (1000)Additionally, with the implementation of proposed changes to Netsmart, our goal is to link backto the referring Episode and Program in Cal-PM and the Client’s associated Financial Eligibilityto determine which Guarantors are claimed for services in our step-down billing model for endof year reconciliation.Contracting Provider SetupPlacer County HHS has registered 1,026 providers in AVATAR Cal-PM to provide BehavioralHealth services. Of these Providers, approximately 70% would be entered into AVATAR MSOfor use with the Care Authorizations. The remaining 30% of these providers are internalproviders who bill for services rendered through Cal-PM. Key facts about our Providers in Cal-PM: • A Provider may be a Primary Care Physician (Physical Health) who provides Behavioral Health services • A Provider may provide services to clients in both Physical Health and Mental Health • A Provider may perform services in multiple programs across Placer’s system of care (MH, AOD, and Physical Health) • A Provider may hold multiple licenses • A Provider may be contracted under one license type (MFT) at one rate and different license type (LCSW) at another rate • A Provider may have multiple reporting numbers depending on Guarantor and Program • A Provider may be contracted at a given rate for all services • A Provider may negotiate an exception rate for a specific client or for all clients from a point in time forward. Placer County – Health & Human Services – Management Information Systems 21 of 31
  22. 22. The above list is key in the accurate mapping of Contracting Providers in MSO to the appropriatePrograms in Cal-PM.Provider Fee Definition SetupProvider Fee Definition is set up for all providers registered in AVATAR MSO. Fees aredefined for Providers based on contracted rates which are standard for all providers of a specificLicense Type for a Funding Source and Plan. Placer County has opted to use the Fee Override inCal-PM so that Fee Definition for services provided in MSO will be the fee posted for theservices to Cal-PM on the client Ledger. Key Facts - Provider Fees: • A provider may be registered to multiple Funding Sources in AVATAR MSO. • A provider may have multiple rates defined for a single Funding Source and CPT Code based on multiple License Type and Level of Care combinations.MSO to Cal-PM System Integration Mapping • MSO Providers are linked to Cal-PM Providers for Medi-Cal Claiming and other State Reporting. • Services are linked between the two products. • Admission/Discharge/Batch Processing to Cal-PM o This option is not used in Placer County. Our business process dictates that a client must be admitted to a program in an episode in Cal-PM prior to having a service authorization created in MSO. As part of this business process, demographic, CSI, Cal-OMS, and other state required data elements are all collected and reported out of Cal-PM. Cal-PM is the system of record to meet all Placer County reporting requirements. The use of this option would not allow us to collect all required information at the time of admission into a managed care Episode/Program.MSO ChallengesIssues in ImplementationChallenge #1: Provider payments from AVATAR MSO are paid out of specific budget accounts that are determined by the type of contracting provider, licensure, services provided, adult or child clients, and the contract details specified in our Performance Accounting System (PAS) used by the Auditor/Controllers Office, who processes the actual payments approved through AVATAR MSO. Due to the nature of the business of managing services to clients to outside network providers, contracts for those providers are somewhat fluid. Amendments to contracts are made on a regular basis to adjust dollars allocated to providers through their contracts, especially towards the end of the fiscal year. To accommodate these changes, we need the ability to manage the authorization of services by providers to each of clients by the dollar amount allocated to a particular budget account. This includes the ability to designate a specific budget account on the Contracting Provider Service Authorizations as well. Placer County – Health & Human Services – Management Information Systems 22 of 31
  23. 23. In short, what this entails is extensive development by our vendor for an optimal solution with full integration between our MSO module and our general ledger accounting system to exchange data on provider contracts and balances. As part of our solution, we have opted to develop our own interface rather than having our vendor work with the Auditor/Controllers vendor to develop the interface. This is not the optimal solution since our locally developed interface is a one-way interface for submitting payment authorizations for check disbursement and does not dynamically manage contract balances to outstanding authorizations.Issue #1: AVATAR MSO Providers cannot be mapped to a Cal-PM Staff member and only one Program due to our current business model of an integrated Health and Human Services agency. Placer County Providers render services under multiple programs. Integration between the two modules should be done at the Service Authorization to Episode/Program to ensure accurate posting of services upon claim adjudication. See background information in preceding sections for examples of Placer’s complex Program, Guarantor, and Provider configurations.Issue #2: In AVATAR MSO, the Service Authorization screen does not allow for workflow routing of a Service Authorization from Pending status to Approved. Only MSO Users with a specific Position Classification should be authorized to Approve Service Authorizations. For Placer County, it is against policy for a Case Manager to both create and Approve a Service Authorization. All Service Authorizations must be Approved by a Supervisor/Manager/Director for services to be legitimately billed against the Authorization. Functionality similar to that which is contained in the AVATAR CWS module must be implemented in MSO prior to moving to a completely paperless system.Issue #3: Conversion of data from an internally developed MS Access database application has proven to be difficult. Data structures in the MS Access database have been modified to an extent that it is no longer viable to attempt a data conversion from the existing system to AVATAR MSO. This is the down side of developing systems internally. Placer County – Health & Human Services – Management Information Systems 23 of 31
  24. 24. AVATAR CWSTo better understand Placer’s plans for implementation of AVATAR CWS, it is helpful tounderstand the basic functionality of the AVATAR CWS module. Below you will find a briefdescription of AVATAR CWS as provided by Netsmart Technologies in the section titled,AVATAR CWS Overview. Additional information on Netsmart’s products can be found Following the overview of the AVATAR CWS module are sections addressingPlacer’s preliminary plans for a phased in implementation of this module as well as someanticipated challenges and issues that will need to be addressed prior to and throughout theimplementation.AVATAR CWS OverviewAVATAR Clinician Workstation (CWS), also known as electronic medical record software,integrates the clinical tools necessary for an interdisciplinary approach to the delivery of healthand human services, including: Treatment/Care Planning, Individual and Group Progress Notes,Assessments, Nursing Reports, and Workflow Management. In addition, AVATAR CWSsupports electronic views of laboratory results and other interdepartmental data and has optionalfeatures, such as MDS entry/tracking, order entry and pharmacy integration (via third-partyinterface).Treatment/Care PlanningEach facility can create care planning templates tailored to address the unique needs of all targetpopulations, supporting best practice guidelines. Keyword searches trigger dictionary statements.The planner steps the clinician through the process from problem definitions and diagnosis togoals, objectives and interventions. There is also an area to record the staff participating in theplan. The system provides the capability to include site-specific information, within theTreatment Plan.Progress NotesProgress notes are entered by the clinician for an individual or group. Ambulatory notes arelinked to outstanding services, or the clinician may post a service directly from the entry of thenote. For a group, a template is used to outline the general group content. Then, for each client,the general outline is individualized.With the use of a mouse, the individual care plan components (Diagnosis, Problems, Goals,Interventions and Objectives) are tagged to their associated progress notes, as required byJCAHO guidelines.Note ReviewThe co-signature feature displays the entire history of notes, particular note types, or specialtyarea notes for review and signature by authorized personnel. The filing of a note can be generalor within a specialty area, (social work, psychiatry, etc.) making note review simple and more Placer County – Health & Human Services – Management Information Systems 24 of 31
  25. 25. direct. The type of note establishes co-signature requirements. Dated addenda are attached toprogress notes directly.DiagnosisDiagnosis information is always available and accessed through the use of a DSM-IV look-uptable. This table supports searches by partial description or code. Multiple selections aresupported for all five axes. All DSM-IV codes are cross-walked to ICD-9. Diagnosticinformation is linked to the treatment/care planner.Workflow NotificationsAVATAR CWS has a fully integrated and user-definable Workflow Notification module thatinforms a clinician when a progress note is required, an assessment requires approval, a progressnote requires a co-signature or a treatment plan review date is approaching. User defined formsare linked to these Notifications.AssessmentsAVATAR CWS is delivered with a battery of behavioral and mental health assessments,including Medical History, Psycho-Social and Nursing evaluations. Additionally, with themodeling tools included with AVATAR CWS, each facility can create an unlimited number ofassessment forms that reflect facility-specific content.Form ModelingModeling tools are available for constructing dictionaries, screens, multi-page forms, tabs anddrop-down menus. Logic is easily incorporated between fields for the more advancedrequirements. Modeling is automatically mapped to the SQL database for access with industrystandard report writers. Any changes made with these tools are compiled so that they areunaffected by periodic updates and enhancements released by Netsmart.ReportsA full reporting suite is available and includes: Missing Patient Flow Sheets, Chart Review,Nursing Administration Summary, Discharge Report and more. For each staff, a To-Do-Listserves as a reminder of needed progress notes, treatment plan reviews and other requireddocumentation. Supervisors have access to staff To-Do-Lists.Treatment Planning Library VersatilityThe clinician can create customized treatment planning libraries or use an optional fullcomplement of Wiley Treatment Planning Libraries, to substantially reduce the preparation timeof individual treatment plans. Library creation is accomplished with a straightforward two-stepprocess. First, utilize the Treatment Planning Dictionary Maintenance option to create a databasethat contains Problems, Problem Definitions, Suggested Diagnoses, Goals, Objectives andInterventions. Then, use the Treatment Planning Library Definition option to create the user-defined library that connects each of the components. An option is also included that copiesProblems (with associated diagnoses, definitions, goals, objectives and interventions) from anylibrary to any user-defined library. Placer County – Health & Human Services – Management Information Systems 25 of 31
  26. 26. SummaryAVATAR CWS is designed to support the multi-disciplinary electronic health record. It bringsthe electronic medical record to the care giver, supporting an integrated view to criticalinformation.Placer’s Scope of AVATAR CWS ImplementationThe initial scope of the AVATAR CWS implementation will be identified for a small test groupmade up of the MHSA staff and the clinical documentation required by both the Mental HealthPlan and Placer’s MHSA Plan. This small test group will consist of approximately 20 staffmembers who will assist in the thorough integration configuration and testing of this module forthe benefit of both program management and IT project staff. Following the successfulimplementation of this small test group, we will move forward with a phased implementation forbusiness areas identified by types of programs defined in AVATAR Cal-PM.Having already reviewed Yolo and Imperial County’s implementation of AVATAR CWS, wehave begun to formulate our plans for implementing AVATAR CWS in Placer County. As partof this pre-planning effort we are in process of scheduling several additional site visits to otherAVATAR CWS users within the state to leverage their knowledge and experience with theproduct to broaden our exposure of how this module can be implemented and to what extent.AVATAR CWS ChallengesIssues in ImplementationChallenge #1 – Paradigm ShiftInternal to Placer County, we expect some push back from clinicians in trying to implementAVATAR CWS because of their current lack of use of a computer system to perform their dutiesas a clinician. Current processes for clinicians are primarily paper based processes for updatedclient records and charts. Our plan is to introduce AVATAR CWS to small workgroups over aperiod of time, providing extensive one-on-one support to ensure the successful transition of to apaperless system. The first group to be implement will be the MHSA staff since they will havealready begun to enter their own FSF, KET, and QTR forms online for a small targetedpopulation; allowing them more time to ensure accurate tracking of client information.Challenge #2 – Continually Evolving System of Care ProcessesWith our system of care processes continually evolving, we need to have an equally dynamicinformation system that can handle changes and at the same time maintain the integrity of theclient’s medical record.Challenges #3 – Evolving California Department of Mental Health RequirementsStandards currently under development by various standards groups and the state willundoubtedly affect our implementation of this module.Over the past several years Netsmart has met all changes in CADDSCal-OMS reporting,CSIDIG reporting, MHSA reporting, and MMEF file format, to meet state requirements. I am Placer County – Health & Human Services – Management Information Systems 26 of 31
  27. 27. confident that they will continue to meet the evolving requirements set forth by the state incurrent and future initiatives.Overall Implementation ConsiderationsAs we have surpassed a major milestone with the implementation of our Practice Managementsystem for Behavioral Health, it was important to take the time to re-orient by reviewing whatthe original project goal was, what we have done to meet that goal, and what still needs to beaccomplished to ensure a complete and successful implementation to meet the organizationalrequirements.Through this project we have learned several things about the overall implementation of ainformation system of this size and scope and we have shared some specific issues that we havehad to address along the way. Some of these lessons learned may be specific to Placer Countywhile others may be more generally applied to any implementation of a health informationsystem. What we would like to offer are key considerations to address in the selection process ofa system that best meets your requirements and the position we took with regards to theseconsiderations.Buy vs. BuildWhile for the most part, Placer HHS has moved to using Commercial Off-The-Shelf (COTS)software for our primary health information systems; there are currently a small number ofsystems that have been developed internally to meet specific requirements. However, the goal isto eventually move all client based health information into one of our HIPAA compliant COTSsystems. To accomplish this, Placer needed a system that would meet the business needs asdefined in the functional requirements as well as having the ability to grow and be customized tomeet the business needs of the future that may be specific to only Placer County. Below is agraphical representation of this spectrum. At one end is the totally custom built system that mayor may not adhere to various standards such as health industry standards, HL7, HIPAA, ANSI,SQL, and security standards. On the other end of this spectrum is the highly standardized COTSsystem that meets all of the aforementioned requirements. To this extreme a standard COTSsystem is inflexible and rigid, imposing a single way of doing business, which may not be theoptimal solutions we all require in this dynamic and ever changing healthcare environment. Highly Configurable/Customizable Custom Built COTS Systems COTS SystemIt is important to determine how import it is to customize an “off-the-shelf” product. Tailor-made solutions can lead to isolation from other customers and user groups. In this same respect,customization of a COTS system may benefit other counties through sharing of customizedscreens, report development, and other value-added enhancements. Placer County – Health & Human Services – Management Information Systems 27 of 31
  28. 28. For Placer, a factor in the decision to select AVATAR was the fact that screens could easily bedeveloped within the AVATAR to capture site specific information while maintaining theintegrity of the data that the custom data fields relate to; whether it client records, practitionerrecords, program records, or any number of other record types contained within AVATAR.National and California Based AVATAR User GroupsNetsmart sponsors both a California based user group as well as a national user group. Whilethis may not sound different from what other vendors do, what sets this user group apart fromothers is the availability of Netsmart resources at each user group meeting.For the California AVATAR User Group, Netsmart regularly host this two day user groupconference three times a year at locations determined by the current user group members. Thisuser group conference consists of Fiscal, Clinical, Administration, and IT staff at all levels oforganizations that currently use AVATAR within California. Additionally, at each meeting,Netsmart coordinates the required company resources to address training needs, new productdemo requests, new state issues and requirements, future development, and product changerequests from the user group. Over the past several years Netsmart has done this by providingregular access to their California based Project Managers, California Support Team, and Sales inaddition to regular participation by Netsmart Management from Customer Support and ProductDevelopment. Through this forum, a number of invaluable enhancements have been presentedand incorporated into the various modules of the AVATAR product.Other national user groups exist that focus on functional areas such as billing, managed care, orelectronic medical records to share different ways of doing business and how the system worksin different environments. At the national level, Netsmart hosts a user conference whereextensive training is provided in all aspects of their products, not only AVATAR. This year willbe the first year Placer participates in this conference.Vendor ParticipationAs detailed above, there are a number ways that your vendor can work with you to foster goodcommunication and evolve with the changing requirements of the environment we work and livein. It is important for a vendor to actively participate in the process with their customers toclosely understand the requirements that they are being asked to meet. In doing so, a vendor canrespond in a timely manner to development requirements ensuring that all imposed deadlines,state and federal, will be met.To ensure that they stay on top of their customers ever-changing requirements, Netsmart takes amulti-faceted approach to working with their customers and understanding their requirements.During implementation, a project manager is assigned from Netsmart to work through the detailsof the implementation with the customers. Following, implementation Netsmart follows upregularly on a weekly basis with the county to address change requests, development in progress, Placer County – Health & Human Services – Management Information Systems 28 of 31
  29. 29. or issues that may arise during the normal course of business. This includes training on newenhancements and features delivered through product updates or version upgrades. Before,during, and after implementation, Netsmart representatives are actively engaged at the state andfederal level ensuring that their product is evolving to meet the changing requirements or theircustomers.All of the above are first hand observations and assessments of Netsmart over the past severalyears. The overall relationship between Netsmart customers and Netsmart has evolved from avendor-customer relationship to more of a business partner relationship that benefits all involved.What Have We Learned From This Adventure? • As in most things, the journey has been just as interesting as the destination. • Planning and communicating are the top items to ensure you do. For every thing you don’t plan or communicate, you pay for 10 times over in lost time, wasted resources, and confused staff. • Our bureaucracies will always extend the length of even a simple task. • Language and wording in written form is critical and oftentimes misinterpreted. Think how others may view your words. • The end users and stakeholders need to participate in the development process to make the final product user-friendly and acceptable. • Other Counties’ pain is your gain -- use their experience to your benefit. • Every software feature will have relative importance to you -- think about it in relationship to other features, and assign it a priority. • Remember that costs can be one-time and on-going -- also stated and hidden. • Your IT staff already have technical skills. What they need to help the effort be successful is more familiarity with the details of the “how” and “why” of the operation -- their natural talent to analyze should be used when documenting your business processes and in the implementation of those business processes in the information system. • It’s important to have a small, committed group of zealots who will talk with each other and figure out how to move the process along, regardless of any temporary roadblocks. • Know what’s going on in the software “biz”. Keep on top of company mergers, failures, victories, and departures of key staff. Placer County – Health & Human Services – Management Information Systems 29 of 31
  30. 30. • Determine how important it is to customize an “off-the-shelf” product -- tailor-made solutions can also lead to isolation from other customers and user groups.• We in California have some unique issues and data needs. It’s important that vendors understand them as much as we do, to make the vendor-customer relationship a true partnership in finding solutions.• Make sure to anticipate the inevitable turf battles that come with any joint effort.• Integration is a word that means different things to different Counties -- figure out what it means to you and if the vendors you consider would enhance it or impede it.• Be familiar with the direction of the software industry, so that you take advantage of forward thinking software -- your vendor decision will be a part of your organization for years to come.• If the vendor can’t explain his or her software to you in understandable language, how do you think it will be received by the “the troops”? This is less of a technical decision than it is an organizational one.• Make sure you have the interest, time, and resources to see this thing through.• Look for the humor and irony -- it’s all around you! Placer County – Health & Human Services – Management Information Systems 30 of 31
  31. 31. Contact InformationFor additional information on Placer’s implementation of AVATAR for Behavioral Health,please feel free to contact Eric Duran at: Eric P. Duran Sr. Technology Solutions Analyst SOC & Health Information Systems Placer County, HHS-MIS 379 Nevada Street Auburn, CA 95603 Office: 530.886.1809 Email: Website: additional information on AVATAR or any other general product information, please feelfree to contact Tim Agar at: Tim Agar Netsmart Technologies Office: 916.536.0360 Email: Website: Placer County – Health & Human Services – Management Information Systems 31 of 31