Defining and benchmarking open problems in single-cell analysis

With the growing number of single-cell analysis tools, benchmarks are increasingly important to guide analysis and method development. However, a lack of standardisation and extensibility in current benchmarks limits their usability, longevity, and relevance to the community. We present Open Problems, a living, extensible, community-guided benchmarking platform including 10 current single-cell tasks that we envision will raise standards for the selection, evaluation, and development of methods in single-cell analysis.

• Luecken, M. D.
• Gigante, S.
• Burkhardt, D. B.
• Cannoodt, R.
• Strobl, D. C.
• Markov, N. S.
• Zappia, L.
• Palla, G.
• Lewis, W.
• Dimitrov, D.
• Vinyard, M. E.
• Magruder, D. S.
• Andersson, A.
• Dann, E.
• Qin, Q.
• Otto, D. J.
• Klein, M.
• Botvinnik, O. B.
• Deconinck, L.
• Waldrant, K.
• Bloom, J. M.
• Pisco, A. O.
• Saez-Rodriguez, J.
• Wulsin, D.
• Pinello, L.
• Saeys, Y.
• Theis, F. J.
• Krishnaswamy, S.