Be the first to like this
Big Data in a neurophysiology research lab… what? by Max Novelli
At RNEL, we have been working hard to lay the foundation to better manage our data and be able to integrate big data and AI technologies into our data management and analysis pipelines. These needs have arisen from the very size of the experimental data that push the limits: they are simply becoming unmanageable even on powerful workstations. We also determined that better query methodologies, validation and visualization tools are needed.
Our long term goal is finding the answer to the following question: Will we ever be able to go from experimental raw data to query curated data with a simple SQL-like language without spending humongous resources and manpower, while using a process that is organic, intuitive and flexible? Can we also leverage modern big data technologies and data science to achieve our goal?
This presentation is the story of an inter-disciplinary journey that started approximately 5 years ago. The journey enabled us to build a deeper knowledge of our data, a better system of management methodologies, as well as tools that allow us to query and aggregate across various datasets and easily improve such functionalities.
In this presentation, we will provide a general background of the work that we do in our lab. First, we will provide some examples of experiments that we conduct as a context in which to explain the data that are acquired and the challenge that comes with them. Next, we will outline some of the questions that researchers asked (and keep asking) when they attempt to work with large data structures to answer their own scientific questions without having to be bogged down by the technologies used and the original format of the data. Finally, we will raise some questions related to data management, which will help to improve validation and reduce the manpower necessary to curate the data. From the big picture, we will walk through the decisions and requirements that came out of our brainstorm sessions and show how far along we currently are in our journey and the path we took to get here. We will conclude by highlighting some of the amazing results that we were able to achieve, such as activation maps and central nervous system stimulation counts.