STAIR Captions

Download

Download dataset from GitHub

Annotation Format

STAIR Captions dataset is provided as JSON files. The annotation format of STAIR Captions follows the one of MS-COCO: STAIR Captions は MS-COCO のアノテーションファイル同様に JSON で配布しています。
JSON のフォーマットは以下の通りです。基本的には MS-COCO のファーマットにそろえています。

annotation{ "id" : int, "image_id" : int, "caption" : str, "tokenized_caption": str, }

For the details of the annotation format, please see MS-COCO download page.

Bibliography

If you use STAIR Captions dataset, please cite the following paper: STAIR Captions をご利用になる際には、以下の論文をご引用いただけると幸いです。

@InProceedings{Yoshikawa2017, title = {STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset}, booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)}, month = {July}, year = {2017}, address = {Vancouver, Canada}, publisher = {Association for Computational Linguistics}, pages = {417--421}, url = {http://www.aclweb.org/anthology/P17-2066} }

Publications

- Yuya Yoshikawa, Yutaro Shigeto, and Akikazu Takeuchi, “STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset”, Annual Meeting of the Association for Computational Linguistics (ACL), Short Paper, 2017.
[ pdf | arXiv | poster ]

In recent years, automatic generation of image descriptions (captions), that is, image captioning, has attracted a great deal of attention. We particularly consider generating Japanese captions for images. Most studies on image captioning target English language, and there are few image caption datasets in Japanese. To tackle this problem, we construct a large-scale Japanese image caption dataset based on images from MS-COCO. Our dataset consists of 820,310 Japanese captions for 164,062 images. In the experiment, we show that a neural network trained with our dataset can generate more natural and better Japanese captions, compared to those generated using English-Japanese machine translation after generating English captions.