| Paper Abstract and Keywords |
| Presentation |
2025-02-18 12:50
A Note on Interpretability of Visual Language Model by Few-shot Learning based on the Linear Representation Hypothesis
Hiroki Okamura, Keisuke Maeda, Ren Togo, Takahiro Ogawa, Miki Haseyama (Hokkaido Univ.) |
| Abstract |
(in Japanese) |
(See Japanese page) |
| (in English) |
Visual language models (VLMs), pre-trained on vast amounts of web-based images and text, have demonstrated impressive zero-shot image classification performance on novel classes.Recently, few-shot learning methods have been proposed to improve the performance of pre-trained VLMs with only a few images. However, these methods lack interpretability and cannot understand the features of the data captured by the model.In this paper, we propose a few-shot learning method based on the linear representation hypothesis, which asserts that the representations obtained from models can be decomposed into a linear combination of multiple elements. The proposed method optimizes and decomposes vectors that are linearly added to class-representing vectors, enabling the interpretation of concepts that the model appends to classes during few-shot learning. Through extensive experiments, we demonstrate that the proposed method enhances the image classification performance of VLMs across 8 datasets while also facilitating the interpretability of the data features captured by the model. |
| Keyword |
(in Japanese) |
(See Japanese page) |
| (in English) |
Visual language models / Few-shot learning / Image classification / Interpretability / / / / |
| Reference Info. |
ITE Tech. Rep., vol. 49, no. 4, ME2025-7, pp. 34-39, Feb. 2025. |
| Paper # |
ME2025-7 |
| Date of Issue |
2025-02-11 (MMS, ME, AIT, SIP) |
| ISSN |
Online edition: ISSN 2424-1970 |
| Download PDF |
|
| Conference Information |
| Committee |
ME AIT MMS IEICE-IE IEICE-ITS SIP |
| Conference Date |
2025-02-18 - 2025-02-19 |
| Place (in Japanese) |
(See Japanese page) |
| Place (in English) |
Hokkaido Univ. |
| Topics (in Japanese) |
(See Japanese page) |
| Topics (in English) |
Image Processing, etc. |
| Paper Information |
| Registration To |
ME |
| Conference Code |
2025-02-ME-AIT-MMS-IE-ITS-SIP |
| Language |
Japanese |
| Title (in Japanese) |
(See Japanese page) |
| Sub Title (in Japanese) |
(See Japanese page) |
| Title (in English) |
A Note on Interpretability of Visual Language Model by Few-shot Learning based on the Linear Representation Hypothesis |
| Sub Title (in English) |
|
| Keyword(1) |
Visual language models |
| Keyword(2) |
Few-shot learning |
| Keyword(3) |
Image classification |
| Keyword(4) |
Interpretability |
| Keyword(5) |
|
| Keyword(6) |
|
| Keyword(7) |
|
| Keyword(8) |
|
| 1st Author's Name |
Hiroki Okamura |
| 1st Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 2nd Author's Name |
Keisuke Maeda |
| 2nd Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 3rd Author's Name |
Ren Togo |
| 3rd Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 4th Author's Name |
Takahiro Ogawa |
| 4th Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 5th Author's Name |
Miki Haseyama |
| 5th Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 6th Author's Name |
|
| 6th Author's Affiliation |
() |
| 7th Author's Name |
|
| 7th Author's Affiliation |
() |
| 8th Author's Name |
|
| 8th Author's Affiliation |
() |
| 9th Author's Name |
|
| 9th Author's Affiliation |
() |
| 10th Author's Name |
|
| 10th Author's Affiliation |
() |
| 11th Author's Name |
|
| 11th Author's Affiliation |
() |
| 12th Author's Name |
|
| 12th Author's Affiliation |
() |
| 13th Author's Name |
|
| 13th Author's Affiliation |
() |
| 14th Author's Name |
|
| 14th Author's Affiliation |
() |
| 15th Author's Name |
|
| 15th Author's Affiliation |
() |
| 16th Author's Name |
|
| 16th Author's Affiliation |
() |
| 17th Author's Name |
|
| 17th Author's Affiliation |
() |
| 18th Author's Name |
|
| 18th Author's Affiliation |
() |
| 19th Author's Name |
|
| 19th Author's Affiliation |
() |
| 20th Author's Name |
|
| 20th Author's Affiliation |
() |
| 21st Author's Name |
|
| 21st Author's Affiliation |
() |
| 22nd Author's Name |
|
| 22nd Author's Affiliation |
() |
| 23rd Author's Name |
|
| 23rd Author's Affiliation |
() |
| 24th Author's Name |
|
| 24th Author's Affiliation |
() |
| 25th Author's Name |
|
| 25th Author's Affiliation |
() |
| 26th Author's Name |
/ / |
| 26th Author's Affiliation |
()
() |
| 27th Author's Name |
/ / |
| 27th Author's Affiliation |
()
() |
| 28th Author's Name |
/ / |
| 28th Author's Affiliation |
()
() |
| 29th Author's Name |
/ / |
| 29th Author's Affiliation |
()
() |
| 30th Author's Name |
/ / |
| 30th Author's Affiliation |
()
() |
| 31st Author's Name |
/ / |
| 31st Author's Affiliation |
()
() |
| 32nd Author's Name |
/ / |
| 32nd Author's Affiliation |
()
() |
| 33rd Author's Name |
/ / |
| 33rd Author's Affiliation |
()
() |
| 34th Author's Name |
/ / |
| 34th Author's Affiliation |
()
() |
| 35th Author's Name |
/ / |
| 35th Author's Affiliation |
()
() |
| 36th Author's Name |
/ / |
| 36th Author's Affiliation |
()
() |
| Speaker |
Author-1 |
| Date Time |
2025-02-18 12:50:00 |
| Presentation Time |
15 minutes |
| Registration for |
ME |
| Paper # |
MMS2025-7, ME2025-7, AIT2025-7, SIP2025-7 |
| Volume (vol) |
vol.49 |
| Number (no) |
no.4 |
| Page |
pp.34-39 |
| #Pages |
6 |
| Date of Issue |
2025-02-11 (MMS, ME, AIT, SIP) |