Prediction of Pediatric Sepsis Using a Deep Encoding Network with Cross Features

Abstract

Abstract:Sepsis poses a serious threat to health of children in pediatric intensive care unit. The mortality from pediatric sepsis can be effectively reduced through in-time diagnosis and therapeutic intervention. The bacilliculture detection method is too time-consuming to receive timely treatment. In this research, we propose a new framework: a deep encoding network with cross features (CF-DEN) that enables accurate early detection of sepsis. Cross features are automatically constructed via the gradient boosting decision tree and distilled into the deep encoding network (DEN) we designed. The DEN is aimed at learning sufficiently effective representation from clinical test data. Each layer of the DEN filtrates the features involved in computation at current layer via attention mechanism and outputs the current prediction which is additive layer by layer to obtain the embedding feature at last layer. The framework takes the advantage of tree-based method and neural network method to extract effective representation from small clinical dataset and obtain accurate prediction in order to prompt patient to get timely treatment. We evaluate the performance of the framework on the dataset collected from Shanghai Children’s Medical Center. Compared with common machine learning methods, our method achieves the increase on F1-score by 16.06% on the test set.

Key words:pediatric sepsis, gradient boosting decision tree, cross feature, neural network, deep encoding network with cross features (CF-DEN)

摘要：脓毒症严重威胁儿科重症监护病房儿童的健康。通过及时诊断和治疗干预，可有效降低小儿脓毒症的死亡率。杆菌培养检测法耗时长，无法及时治疗。提出了一种新的框架：一个具有交叉特征的深度编码网络（CF-DEN），可以准确地进行早期脓毒症的检测。通过梯度提升决策树自动构造交叉特征，并提取到所设计的深度编码网络（DEN）中。DEN旨在从临床试验数据中学习足够有效的表征。DEN的每一层通过注意力机制对当前层计算中涉及的特征进行过滤，逐层叠加输出当前预测，得到最后一层嵌入特征。该框架利用基于树的方法和神经网络的方法，从小型临床数据集中提取有效表征，得到准确预测，从而促使患者得到及时治疗。在上海儿童医学中心收集的数据集上评估了框架的性能。与常见的机器学习方法相比，我们方法在测试集上的F1-score提高了16.06%。

关键词:儿童脓毒症，梯度提升决策树，交叉特征，神经网络，交叉特征深度编码网络

CLC Number:

TP399
R515.3

CHEN Xiao^1,2(陈潇), ZHANG Rui^1,2(张瑞), TANG Xinyi^1,2(汤心溢), QIAN Juan^3∗(钱娟). Prediction of Pediatric Sepsis Using a Deep Encoding Network with Cross Features[J]. J Shanghai Jiaotong Univ Sci, 2024, 29(1): 131-140.

References

[1] FLEISCHMANN C, SCHERAG A, ADHIKARI N K J, et al. Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations [J]. American Journal of Respiratory and Critical Care Medicine, 2016, 193(3): 259-272.
[2] SINGER M, DEUTSCHMAN C S, SEYMOUR C W, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3) [J]. JAMA, 2016, 315(8): 801-810.
[3] DESAUTELS T, CALVERT J, HOFFMAN J, et al. Prediction of sepsis in the intensive care unit with minimal electronic health record data: A machine learning approach [J]. JMIR Medical Informatics, 2016, 4(3): e28.
[4] DESAUTELS T, HOFFMAN J, BARTON C, et al. Pediatric severe sepsis prediction using machine learning [EB/OL]. (2017-11-22). https://www.biorxiv.org/content/10.1101/223289v1.
[5] ZHANG Z H, HONG Y C. Development of a novel score for the prediction of hospital mortality in patients with severe sepsis: The use of electronic healthcare records with LASSO regression [J]. Oncotarget, 2017, 8(30): 49637-49645.
[6] LE S, HOFFMAN J, BARTON C, et al. Pediatric severe sepsis prediction using machine learning [J]. Frontiers in Pediatrics, 2019, 7: 413.
[7] CALVERT J S, PRICE D A, CHETTIPALLY U K, et al. A computational approach to early sepsis detection [J]. Computers in Biology and Medicine, 2016, 74: 69-73.
[8] FUTOMA J, HARIHARAN S, HELLER K. Learning to detect sepsis with a multitask Gaussian process RNN classifier [C]//34th International Conference on Machine Learning. Sydney: ICML, 2017: 1174-1182.
[9] FUTOMA J, HARIHARAN S, HELLER K. An improved multi-output Gaussian process rnn with realtime validation for early sepsis detection [C]//2nd Machine Learning for Healthcare Conference. Boston: PMLR, 2017: 243-254.
[10] FRIEDMAN J H. Greedy function approximation: A gradient boosting machine [J]. The Annals of Statistics, 2001, 29(5): 1189-1232.
[11] HE X R, PAN J F, JIN O, et al. Practical lessons from predicting clicks on ads at Facebook [C]//Eighth International Workshop on Data Mining for Online Advertising. New York: ACM, 2014: 1-9.
[12] ARIK S O, PFISTER T. TabNet: Attentive interpretable tabular learning [EB/OL]. (2020-12-09). https://arxiv.org/abs/1908.07442.
[13] IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate shift [C]//International Conference on Machine Learning. Lille: PMLR, 2015: 448-456.
[14] DAUPHIN Y N, FAN A, AULI M, et al. Language modeling with gated convolutional networks [C]//International Conference on Machine Learning. Sydney: PMLR, 2017: 933-941.
[15] MARTINS A, ASTUDILLO R. From softmax to sparsemax: A sparse model of attention and multilabel classification [C]//International Conference on Machine Learning. New York: PMLR, 2016: 1614- 1623.
[16] FLEMING S, THOMPSON M, STEVENS R, et al. Normal ranges of heart rate and respiratory rate in children from birth to 18 years of age: A systematic review of observational studies [J]. The Lancet, 2011, 377(9770): 1011-1018.
[17] CHAWLA N V, BOWYER K W, HALL L O, et al. SMOTE: Synthetic minority over-sampling technique [J]. Journal of Artificial Intelligence Research, 2002, 16: 321-357.
[18] HOCHREITER S, SCHMIDHUBER J. Long shortterm memory [J]. Neural Computation, 1997, 9(8): 1735-1780.
[19] RUMELHART D E, HINTON G E, WILLIAMS R J. Learning internal representations by error propagation [M]//Parallel distributed processing: Explorations in the microstructure of cognition: Foundations. Cambridge: MIT Press, 1987: 318-362.
[20] CRAMER J S. The origins of logistic regression [EB/OL]. (2003-01-25). https://ssrn.com/abstract=360300.
[21] CORTES C, VAPNIK V. Support-vector networks [J]. Machine Learning, 1995, 20(3): 273-297.
[22] ALTMAN N S. An introduction to kernel and nearestneighbor nonparametric regression [J]. The American Statistician, 1992, 46(3): 175-185.
[23] HO T K. Random decision forests [C]//3rd International Conference on Document Analysis and Recognition. Montreal: IEEE, 1995: 278-282.
[24] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[25] VAN DER MAATEN L. Accelerating t-SNE using tree-based algorithms [J]. Journal of Machine Learning Research, 2014, 15(1): 3221-3245.
[26] BORG I, GROENEN P. Modern multidimensional scaling: Theory and applications [J]. Journal of Educational Measurement, 2003, 40(3): 277-280.
[27] TIPPING M E, BISHOP C M. Probabilistic principal component analysis [J]. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 1999, 61(3): 611-622.