DB Development work for Association of General Contractors 5394 Engineering Database Design Fall 2011, Dr. Weihang Zhu Project team: Vaibhav Chauhan, Fatemeh Hosseinzadehdastak, Ashish Patel
Presentation Outline1. Project Description a) What is the objective of your database? b) Who is your customer? c) What is the requirement from your customer? d) Has your application been reviewed and approved by your customer? e) What have you done for the project overall?2. Overview description of the system’s operation and workflow3. E-R Diagram4. Relational Schema5. Lists of the detailed Database Structure a) Tables b) Queries6. Technical problems7. Conclusions and future work8. Acknowledgment9. References
Project Description - ObjectiveO Provide an application that can be used by the customer to meet their business needs efficiently, and in a robust manner.O To challenge ourselves further, we parsed the customer’s existing, unstructured data (found to be in first-normal form), into SQL commands.
Project Description - CustomerO For this project, we reached out to a local membership-based trade organization that has national presence.O The Association of General Contractors founded in 1918, and currently boasts approximately 33,000 member firms.O Our customer, AGC of Southwest Texas, was charted in 1940 and currently has 278 member companies. The AGC’s Director, Mike White, is a believer in adopting technology in the construction industry.
Project Description - RequirementsThe member companies of the AGC pay an annual membership fee, resulting theAGC’s need to track payment dates, and use this information to generate a) An accurate list of current members (for the dissemination of information to members only), b) A list of pending membership expirations based on last-payment date, c) A member’s payment activity, and lastly d) A list of inactive memberships .Additionally, the AGC wanted the capability to search their database by a) Individual name b) Telephone number c) Fax number d) UCI type: Each company in the database is associated with one or many UCI codes that detail the specialties of that company. For example, HB Nield & Sons features UCI codes of 22222, 22234, 23423, 23425, meaning they specialize in various electrical skills.
Project Description - CustomerO Mr. White commissioned similar work in the recent past.O Because of the low quality of this previous work and a lack of documentation, the database was never used by the AGC’s employees. They returned to their Excel spreadsheets.
BeforeSELECT Contacts.Company, Contacts.CategoriesFROM Contacts;Only one ContactsQuery and one Table!Cost AGC Paid to Private Contractor in 2009: $1,500.00
Membership ManagementO 90% Customer Relationship Management (CRM with membership and transaction logging).O 10% Other Contacts (non-member): O Mayor of Beaumont O President of Chamber of Commerce O Public Figure 3 O Beaumont Hotel 1
Project Description – What we did!1. After analyzing the raw database, we provided the ER diagram2. Relational Schema3. Normalization4. We provided the Access database5. We developed a GUI build in Visual Studio using VB.NET
Project Description - How we did it!O We began the process by reviewing the 52 columns of the member data provided initially by Mr. White, in first-normal form database.O Next, the exclusion of some unnecessary and often unused fields such as HomeTel#3 and CarPhone#, resulting in 26 columns exported to a CSV.O Then we imported the raw data into Python 3.2.2 to perform a parsing operation that resulted in over 2600 SQL INSERT statements.O Access cannot execute multiple SQL statements in a batch, and this was unknown to us at the time. Fortunately, Dr. Zhu, suggested using MySQL as a temporary storage space for the data, and exporting the structured data for import into Access. This process was ultimately successful, thanks to Dr. Zhu, and provides the AGC with a significant time savings.O We designed intuitive Windows forms that met the data entry and query needs of the AGC. We faced many challenges at this step.
System Operation – How it Works!!O Primarily, the solution is a “transaction-oriented ” database application, however it must act as a “display-oriented” application to deliver customer value.O When a member company pays their annual dues, and analyst enters a transaction record with the current date. SELECT MembershipID, OrganizationID, Expiration_date, [Last Payment Date] FROM tblTransaction WHERE (DateValue([Last Payment Date]) + 365 > NOW())O On a monthly basis, the AGC must send out letters reminding their members to remit membership dues. Using the “Pending expiration” query, they can quickly generate a list of companies which need to be called, mailed, and emailed a reminder. SELECT DISTINCT tblOrganization.Organization_Name, DateValue([Last Payment Date])+365 AS Expr1 FROM tblOrganization INNER JOIN (tblMembership INNER JOIN tblTransaction ON tblMembership.MembershipID = tblTransaction.MembershipID) ON tblOrganization.ORGANIZATION_ID = tblTransaction.OrganizationID WHERE (((DateValue([Last Payment Date])+365)<DateValue(Now())-30 And (DateValue([Last Payment Date]) +365)>DateValue(Now())-90));O If a non-local builder is searching for a local underwater welder , they will contact the AGC in search of a list of members that specifically mention the skill in their profile. Our application permits the AGC to record those skills by adding them to the member organization’s profile through the UCI code form and later return a list of active memberships.
Project Description – Review ProcessO We regularly met or communicated by email with AGC.O Each of the forms delivered in the resulting application have been approved by him.O He wants our team to continue development on the application to meet the needs of the AGC in the future.O Mr. White foresees changes in local legislation that will require the AGC to retain additional data about their members, create filters on this information, and invite members based on their possession or fulfillment of certain criteria.
TablesO The resulting database features 14 tables, 10 of which are potentially static, and 4 of which will be populated with new or edited records as the AGC utilizes the application in their operations.O Category*, Fax_Numbers, Individuals, IndividualSkillsRelation, JobTitle, Membership, Organization, Organization_Addresses, Organization_Individual_Relation, Organization_UCI_Relation, Skills, StatesList, Telephone_Numbers, Transaction, UCIDescriptionO *Category table was added by the customer very late in the process, and is not reflected in the original ER and Relational Table Diagrams.
Query 1All memberships that will expire in 30-90 days (TodayDate-(LastPaid+365 ) < 30)SELECT DISTINCT tblOrganization.Organization_Name,DateValue([Last Payment Date])+365 AS Expr1FROM tblOrganization INNER JOIN (tblMembership INNER JOINtblTransaction ON tblMembership.MembershipID =tblTransaction.MembershipID) ONtblOrganization.ORGANIZATION_ID =tblTransaction.OrganizationIDWHERE (((DateValue([Last Payment Date])+365)<DateValue(Now())-30 And (DateValue([Last PaymentDate])+365)>DateValue(Now())-90));
Query 2What are the email addresses for all active membersSELECT DISTINCT tblOrganization.[E-mail], DateValue([lastpayment Date])+365 AS [Activate Status]FROM tblOrganization INNER JOIN (tblMembership INNERJOIN tblTransaction ON tblMembership.MembershipID =tblTransaction.MembershipID) ONtblOrganization.ORGANIZATION_ID =tblTransaction.OrganizationIDWHERE (((DateValue([last payment Date])+365)>Now()));
Query 3Inactive Members (where TodayDate-(LastPaid+365 ) < 0)SELECT DISTINCT tblOrganization.Organization_Name,DateValue([Last Payment Date])+365 AS Expr1FROM tblOrganization INNER JOIN (tblMembership INNERJOIN tblTransaction ON tblMembership.MembershipID =tblTransaction.MembershipID) ONtblOrganization.ORGANIZATION_ID =tblTransaction.OrganizationIDWHERE (((DateValue([Last Payment Date])+365)>Now()));
Technical Difficulties!!!!The storage of new informationOFirst, we attempted to programmatically access the database , yetoften received an error (“passing null values”) when executing the UPDATEcommand.OAs an alternative, we created a newRow object and when calling thedatatable.Rows.Add(NewRow) function, we experienced a different error.OSome of the errors we received when attempting to programmatically addand edit data where similar to these: O “The record cannot be deleted or changed because table tblFAX_Numbers includes related records.” O “An INSERT INTO query cannot contain a multi-valued field.” O “Reference to a non-shared member requires an object reference.”OInstead, we used Visual Studio TableAdapters to maintain the Connectionobjects. The simplest successful implementation used theTableAdapterManager that was automatically created when dragging atable node from the Data Sources onto a form in Design View.
Technical Difficulties - Python O We took a “crash course” in Python 3.2.2 using resources such as docs.python.org and free training videos readily available on YouTube.com. O We abandoned sub-projects such as parsing the UCI- Organization relation. O cursor positioning, O reading lines versus characters O character substitutions O However, we did successfully generate approximately 2600 lines of SQL INSERT statements using the Python environment. O Access has no method for batch processing these statements, and thanks to Dr. Zhu, we were able to import the large volume of data into our application
The Number of Tables and Forms 18 Tables 19 Forms 115 Button
ReferencesO MSDNO Python.orgO Text Processing with PythonO Developing Web-Enabled Decision Support SystemsO Sourcecodester.comO Stackoverflow.comO YouTube.com – Python Basics, Ep1-4, by ZJunaideenO http://www.penzilla.net/tutorials/python/fileio/
Conclusions & Future WorkO First, the challenges we faced, and the days we endured trying to answer questions from hundreds of blogs and web-how-to’s showed us how prevalent and version-specific Visual Studio and Access are as development environments.O How much having an ER diagram and the relational tables can help us. The practice of normalization can help us to resolve clear relationships between tables and distill binary and ternary relationships, resulting in efficient data storage and makes complex queries more stable.O We fully expect the AGC to demand 100% functionality, and while we feel we are technically able to deliver, we will require outside assistance.O Future development projects in Objective-C and Java this winter.
AcknowledgmentO The teammates for their many 4am nights and dedication to understanding hierarchical data stability, database connections, parsing, creating SQL statements in-code, and teamwork.O Like to acknowledge the patience of Mike White, the Director of the AGC for his co-operation and willingness to provide his time and experience as we learned the business processes involved in operating a membership-based organization.O Dr. Zhu for his lectures and for providing us with the benefit of his experience. We would especially like to thank him for his attention and assistance in solving problems we faced with executing batch commands output by Python in MySQL, and importing the resulting data into Access.O We would like to thank the rest of the class for their communication and assistance throughout the process. Learning from your challenges and solutions gave us a competitive motivation, but helping each other solve problems made us feel our class has the potential to create great solutions.