Multimodal learning: Difference between revisions

Revision as of 21:28, 29 January 2023

Revision as of 06:01, 15 March 2023

Line 83:

==Application==

Multimodal deep Boltzmann machines are successfully used in classification and missing data retrieval. The classification accuracy of multimodal deep Boltzmann machine outperforms [[support vector machine]]s, [[latent Dirichlet allocation]] and [[deep belief network]], when models are tested on data with both image-text modalities or with single modality.{{Citation needed|date=November 2022}}. Multimodal deep Boltzmann machine is also able to predict missing modalities given the observed ones with reasonably good precision.{{Citation needed|date=November 2022}} Self Supervised Learning brings a more interesting and powerful model for multimodality. [[OpenAI]] developed CLIP and [[DALL-E]] models that revolutionized multimodality.

Self Supervised Learning brings a more interesting and powerful model for multimodality. [[OpenAI]] developed CLIP and [[DALL-E]] models that revolutionized multimodality.

Multimodal deep learning is used for [[cancer screening]] – at least one system under development [[Data integration#Medicine and Life Sciences|integrates]] such different types of data.<ref>{{cite news |last1=Quach |first1=Katyanna |title=Harvard boffins build multimodal AI system to predict cancer |url=https://www.theregister.com/2022/08/09/ai_cancer_multimodal/ |access-date=16 September 2022 |work=The Register |language=en}}</ref><ref>{{cite journal |last1=Chen |first1=Richard J. |last2=Lu |first2=Ming Y. |last3=Williamson |first3=Drew F. K. |last4=Chen |first4=Tiffany Y. |last5=Lipkova |first5=Jana |last6=Noor |first6=Zahra |last7=Shaban |first7=Muhammad |last8=Shady |first8=Maha |last9=Williams |first9=Mane |last10=Joo |first10=Bumjin |last11=Mahmood |first11=Faisal |title=Pan-cancer integrative histology-genomic analysis via multimodal deep learning |journal=Cancer Cell |date=8 August 2022 |volume=40 |issue=8 |pages=865–878.e6 |doi=10.1016/j.ccell.2022.07.004 |pmid=35944502 |s2cid=251456162 |language=English |issn=1535-6108|doi-access=free }}