Question icon
Your Current Search
Choose below to refine your search
Research Topic
Download abstract book

Download the NI2012 abstract book here. The page numbers in the index are clickable for easy browsing.


Automatic spike sorting evaluation: A website based community approach


Felix Franke (ETH Zürich, Bio Engineering Laboratory (BEL)), Philipp Meier (School for Electrical Engineering and Computer Science, Berlin Institute of Technology, Germany), Andrey Sobolev (Biologie II, Ludwig-Maximilians-Universität München, Planegg-Martinsried, Germany), Espen Hagen (Department of Mathematical Sciences and Technology, Norwegian University of Life Sciences, 1432 Ås, Norway), Andreas Hierlemann (ETH Zürich, Bio Engineering Laboratory (BEL)), Klaus Obermayer (Bernstein Center for Computational Neuroscience, Berlin, Germany), Gaute T Einevoll (Department of Mathematical Sciences and Technology, Norwegian University of Life Sciences, 1432 Ås, Norway), Thomas Wachtler (Biologie II, Ludwig-Maximilians-Universität München, Planegg-Martinsried, Germany)

The goal of spike sorting is to separate the spike trains of individual neurons from extracellular recordings. This step is crucial in many neuroscientific experiments since one extracellular electrode usually records the activity of several neurons. Despite the large effort to develop automatic algorithms (Lewicki, 1998) to solve the problem, spike sorting can be still considered as well an art as an exact science with a large manual component. For the quantitative evaluation of spike sorting algorithms the ground truth of the data analyzed, i.e. the number of neurons and their firing times, has to be known (Einevoll et al., 2011). Real extracellular recordings provide no suitable benchmark data because of the inherent absence (or at least very limited presence) of ground truth information. Thus, using simulated surrogate data is the traditional way to evaluate spike sorting algorithms. However, most scientists use their own simulated data, making comparisons between different publications very difficult.
Here, we develop a framework for automated spike sorting evaluation based on several different benchmark datasets used in recent publications. The framework is implemented on a website that allows the user to download benchmark files, upload their sorting results, and compare the performance of their sorting algorithm to those of other users. Furthermore, users can also upload their own benchmark datasets and make them available to the community. We hope that the website will help in comparing the performance of different spike sorting algorithms and foster the development of new ones.
The underlying framework, i.e., website frontend and evaluation backend, can be generalized to other, similar, algorithm evaluation problems, such as encountered in EEG data analysis.

The website is available at

Supported by the Research Council of Norway (NevroNor, eScience, Notur), by INCF through its German Node (BMBF grant 01GQ0801), and by the Deutsche Forschungs Gemeinschaft (DFG) with grant GRK 1589/1.

Einevoll, G. T., Franke, F., Hagen, E., Pouzat, C., and Harris, K. D. (2011). Towards reliable spike-train recordings from thousands of neurons with multielectrodes. Current Opinion in Neurobiology 27, 1–7.
Lewicki, M. S. (1998). A review of methods for spike sorting: the detection and classification of neural action potentials. Network (Bristol, England) 9, R53–R78
Automatic spike sorting evaluation: A website based community approach
Usage of the website for automatic spike sorting evaluation. Once a benchmark was uploaded by a user and made available to the community, other users can download the extracellular data (1). Applying a sorting algorithm of their choice (also manual sorting is possible) they will get a sorting result (2) that can be uploaded to the website (3). The website will compute an evaluation of that sorting using the ground truth data, and the results can be inspected by the user (4). If the user wants to compare the result to those of other users, he/she can publish the result (5).
Preferred presentation format: Demo
Why demo: It would give us the possibility to show the functional website and the way it is used. What would be needed is a computer with internet access
Topic: Infrastructural and portal services