Software Defined Media

Object-based digital media

Media data is decoupled from audio-visual data and 3D meta-data, which represents multiple objects that exist in 3D space, and transmitted to remote locations for the reproduction of 3D space.

Software defined media space

The software manages the audio-visual objects in the virtual 3D space and renders images and sounds suitable for the rendering environments.

Internet by design

Targeting new research areas and markets involving Internet-by-design audio-visual environments.

Notices

News

Press Releases

Press Conference

We hold a press conference about LiVRation (remote VR live streaming service).

Ultra-realistic SDM orchestral recordings accessible as open data

We published a press release about Ultra-realistic SDM orchestral recordings

Awards

LODC 2020

We won the first prize in Linked Open Data Challenge 2020 Japan

DICOMO2020

Our paper published in Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2020) won the best presentation award.

DICOMO2019

Our paper published in Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2019) won the Young researcher award.

Yamashita SIG Research Award

Dr. Tsukada (Chair) was awarded Yamashita SIG Research Award.

DICOMO2018

Our paper published in Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2018) won the best paper award.

Live Hackasong 2017

We won the first prize in LIVE HACKASONG hold in Billboard Tokyo.

DICOMO2017

Our paper published in Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2017) won the best paper award.

Billboard LIVE MUSIC HACKASONG

We got the second prize in LIVE MUSIC HACKASONG hold in Billboard Tokyo.

Honor of "Dean's List"

Keiko Ogawa (KMD:Graduate School of Media Design, Keio)was awarded a honor of "Dean's List" upon completion of Master's Degree.

Media

DateTitleType
2018-05-15 NHK、 「VRで音楽ライブ 誰でもどこでも」、ニュースチェック11TV
2018-05-15 時事通信社Newspaper
2018-05-15 日経産業新聞Newspaper
2018-05-15 日刊工業新聞Newspaper
2018-05-15 産経新聞Newspaper
2018-05-14 ビルボード×Cipのハッカソンで誕生した"LiVRation"が実用化に向け記者発表を実施 | Daily News | Billboard JAPAN Web Article
2018-05-14 ライブをVRで鑑賞、視点や音響もカスタマイズできる「LiVRation」 - AV Watch Web Article
2018-05-14 臨場感高くライブを再現するシステム「ライブレーション」:時事ドットコム Web Article
2018-05-14 映像や音、臨場感高く=遠隔地でライブ再現-東大など:時事ドットコム Web Article
2018-05-14 音楽ライブに360度映像で参加 東大がVR技術(動画あり) | 科学技術・大学 ニュース | 日刊工業新聞 電子版 Web Article
2018-05-14 ライブを超えた体験を可能にする遠隔ライブVR配信プラットフォーム「LiVRation」を開発 - 産経ニュース Web Article
2018-05-14 音楽ライブの臨場感をVRで体験 東京・渋谷 | NHKニュース Web Article
2018-02-05 テクノロジーはエンタテインメント体験をどう拡張する? 『LIVE HACKASONG』レポート - Real Sound|リアルサウンド Web Article
2018-02-02 "エンタテインメント体験の拡張"をテーマにしたロングラン・ハッカソン「Live HACKASONG」最終審査結果発表 | Musicman-NET Web Article
2018-02-01 最新技術を使った新たなライブの楽しみ方とは? 【LIVE HACKASONG】最終審査&表彰式レポート | Daily News | Billboard JAPAN Web Article
2017-02-02 "ライブ"と"テクノロジー"のコラボハッカソン開催〜Billboard LIVE MUSIC HACKASONGに垣間見る未来 - WirelessWire News(ワイヤレスワイヤーニュース) Web Article
2017-01-30 "ライブ体験の拡張"を目指すハッカソン、ビルボードライブ東京にて最終審査が開催 | Daily News | Billboard JAPAN Web Article
2017-01-28 "ライブ体験の拡張"をテーマにしたハッカソン「Live Music HACKASONG」最終審査・表彰式を開催 | Musicman-NET Web Article
2016-09-07 【CEDEC2016】ゲーム技術を業界外へ持ち出そう!サウンド技術から切り開く新ビジネスへの挑戦事例 | GameBusiness.jp Web Article
2016-07-22 ASCII.jp:合計82チャンネル、360度4Kカメラでオーケストラを収録 Web Article
Showing 20 out of 20 entries

Events

SDM Symposium 2019

SDM symposium was held in the University of Tokyo.

Interop 2019

We demonstrated our works about Software Defined Media in Interop 2019@Makuhari Messe.

KMD Forum 2018

We presented our prototype systems in the forum of Graduate School of Media Design, Keio University.

Interop 2018

We demonstrated our works in Interop 2018@Makuhari Messe.

Live Hackasong 2017

We won the first prize in LIVE HACKASONG hold in Billboard Tokyo.

KMD Forum 2017

We presented our prototype systems in the forum of Graduate School of Media Design, Keio University.

SDM Symposium 2017

SDM workshop and ideathon were held in the University of Tokyo.

Billboard LIVE MUSIC HACKASONG

We got the second prize in LIVE MUSIC HACKASONG hold in Billboard Tokyo.

SDM Workshop 2016

SDM workshop and ideathon were held in the University of Tokyo.

Recording of the concert of Department of Musicology, Keio University

We recorded the concert of Department of Musicology, Keio University for the future experiments data.

Smartlife Hackathon 2015 Winter

We provided 3D audio management system using SDM in the Smartlife hackathon.

IETF 94 @Yokohama Bits-N-Bites

We displayed the SDM demonstration of the interactive object-based audio-visual service in IETF 94 @Yokohama Bits-N-Bites.

Smartlife Hackathon 2015 Autumn

Smartlife hackathon for organizers and developers was held in the University of Tokyo. We developed the demonstration that combines the indoor localization technique and the object-oriented audio-visual service.

Smartlife Hackathon 2015 Summer

We provided 3D audio management system using SDM in the Smartlife hackathon.

Interop 2015 Tokyo

We displayed the SDM demonstration of the interactive object-based audio-visual service at Interop 2015 Tokyo in Makuhari Messe.

WIDE Camp 2015 Spring

We displayed the SDM demonstration of the interactive object-based audio-visual service in the WIDE camp at Hamanako Royal Hote in Shizuoka.

Members

Hiroshi Esaki
Hiroshi Esaki
Chair
Manabu Tsukada
Manabu Tsukada
Chair

Publications

English Publications

Semantic Digital Twin for interoperability and comprehensive management of data assets

Kazuma Inokuchi, Jin Nakazato, Manabu Tsukada, Hiroshi Esaki

IEEE International Conference on Metaverse Computing, Networking and Applications (IEEE MetaCom 2023), Jun. 2023

inproceedings

ZIGEN: A Windowing System Enabling Multitasking Among 3D and 2D Applications in Immersive Environments.

Akihiro Kiuchi, Taishi Eguchi, Jun Rekimoto, Manabu Tsukada, Hiroshi Esaki

Vancouver, Canada: ACM SIGGRAPH 2022 Posters, 2022

misc
DOI

WebRTC-based measurement tool for peer-to-peer applications and preliminary findings with real users

Kosuke Nakagawa, Manabu Tsukada, Keiichi Shima, Hiroshi Esaki

16th Asian Internet Engineering Conference (AINTEC), Dec. 2021

inproceedings
DOI

Reinforcement Learning Based Optimal Camera Placement for Depth Observation of Indoor Scenes

Yichuan Chen, Manabu Tsukada, Hiroshi Esaki

The 2021 IEEE International Conference on Networking, Sensing and Control (ICNSC), Dec. 2021

inproceedings
DOI

co-Sound: An interactive medium with WebAR and spatial synchronization

Kazuma Inokuchi, Manabu Tsukada, Hiroshi Esaki

The 19th IFIP International Conference on Entertainment Computing (ICEC) 2020, Nov. 2020

inproceedings

Web360²: An Interactive Web Application for viewing 3D Audio-visual Contents

Shin Kato, Tomohiro Ikeda, Mitsuaki Kawamorita, Manabu Tsukada, Hiroshi Esaki

17th Sound and Music Computing Conference (SMC), Jun. 2020, pp. 32–39

inproceedings
DOI

LiVRation: Remote VR live platform with interactive 3D audio-visual service

Takashi Kasuya, Manabu Tsukada, Yu Komohara, Shigeki Takasaka, Takuhiro Mizuno, Yoshitaka Nomura, Yuta Ueda, Hiroshi Esaki

IEEE Games Entertainment & Media Conference (IEEE GEM) 2019, Jun. 2019, pp. 1–7

inproceedings
DOI

The Software Defined Media Ontology for Music Events

Ray Atarashi, Takuro Sone, Yu Komohara, Manabu Tsukada, Takashi Kasuya, Hiraku Okumura, Masahiro Ikeda, Hiroshi Esaki

Workshop on Semantic Applications for Audio and Music (SAAM) held in conjunction with ISWC 2018, Oct. 2018, pp. 15–23

inproceedings
DOI

Software Defined Media: Virtualization of Audio-Visual Services

Manabu Tsukada, Keiko Ogawa, Masahiro Ikeda, Takuro Sone, Kenta Niwa, Shoichiro Saito, Takashi Kasuya, Hideki Sunahara, Hiroshi Esaki

IEEE International Conference on Communications (ICC2017), May 2017, pp. 1–7

inproceedings
DOI

Investigating integration between auditory and visual location information presented in an augmented-reality device

Okumura,Hiraku, Sushrut,Khiwadkar, Kim,Sungyoung

The Journal of the Acoustical Society of America, vol. 140, no. 4, pp. 3277–3277, Oct. 2016

article
DOI

Smartphone-Based 360° Video Streaming/Viewing System including Acoustic Immersion

Kenta Niwa, Daisuke Ochi, Akio Kameda, Yutaka Kamamoto, Takehiro Moriya

Audio Engineering Society Convention 141, Sep. 2016, p. 9676

conference

New Recording Application for Software Defined Media

Masahiro Ikeda, Takuro Sone, Kenta Niwa, Shoichiro Saito, Manabu Tsukada, Hiroshi Esaki

Audio Engineering Society Convention 141, Sep. 2016, p. 9610

conference

Tangible sound : a tangible interface for object-based sound system

Keiko Ogawa

KMD:Graduate School of Media Design, Keio University, 2016

Dean's List Award

mastersthesis

Japanese Publications

IFCモデルサーバを用いたBIM向けWebビューワの実装

古田悟, 中里仁, 塚田学, 江崎浩

マルチメディア,分散,協調とモバイル(DICOMO2023)シンポジウム, Jul. 2023 (Japanese)

conference

5G を用いた遠隔コラボレーションの QoE 評価に関する研究

伊藤広記, 中川紘輔, 中里仁, Romain Fontugne, 塚田学, 江崎浩

マルチメディア,分散,協調とモバイル(DICOMO2023)シンポジウム, Jul. 2023 (Japanese)

conference

メタバース実現に求められるデジタルインフラ技術とその展望.

塚田学

公益財団法人 原総合知的通信システム基金主催 特別セミナー, 2023 (Japanese)

misc

Software Defined Media activity report 2022

SDM consortium member

WIDE Report, Mar. 2023 (Japanese)

techreport

5Gモバイルブロードバンド下での遠隔コラボレーションの実現可能性調査

中川紘輔, 中里仁, ロマンフォンテュニュ, 伊藤広記, 塚田学, 江崎浩

電子情報通信学会 ITS研究会, Feb. 2023 (Japanese)

conference

メタバース実現に求められるデジタルインフラの課題と要件.

塚田学

第6回「Web3時代に向けたメタバース等の利活用に関する研究会」, 2022 (Japanese)

misc

デジタルツインの相互運用性向上に向けた動的事物を記述するオントロジーの設計

井口和真, 塚田学, 江崎浩

人工知能学会:セマンティックウェブとオントロジー研究会, Nov. 2022 (Japanese)

conference
DOI

[招待講演]メタバースのためのメディア相互運用基盤.

塚田学

東京・有明 TFTホール: Archi Future 2022, 2022 (Japanese)

misc

Software Defined Media activity report 2021

SDM consortium member

WIDE Report, Mar. 2022 (Japanese)

techreport

[ビジュアル版]バーチャル・リアリティ百科〜進化するVRの現在と可能性〜.

インディラ・トゥーヴェニン(著), ロマン・ルロン(著), 大塚宏子(訳), 塚田学(日本語版監修), 水野拓宏(用語監修)

原書房, 2021 (Japanese)

book

生産管理のためのデジタルツインアプリケーション

川村地平, 青谷和真, 藤橋貞光, 稗田啓介, 中島高英, 塚田学, 江崎浩

マルチメディア,分散,協調とモバイル(DICOMO2021)シンポジウム, Jun. 2021 (Japanese)

conference

デジタルツイン基盤とその3Dデータ活用

青谷和真, 川村地平, 塚田学, 落合秀也, 江崎浩

マルチメディア,分散,協調とモバイル(DICOMO2021)シンポジウム, Jun. 2021 (Japanese)

conference

WebRTC を用いた宅内環境間 P2P ラウンドトリップタイム計測

中川紘輔, 塚田学, 落合秀也, 江崎浩

マルチメディア,分散,協調とモバイル(DICOMO2021)シンポジウム, Jun. 2021 (Japanese)

conference

Software Defined Media activity report 2020

SDM consortium member

WIDE Report, Mar. 2021 (Japanese)

techreport

co-Sound: The interactive media platform with Web AR

Kazuma Inokuchi, Manabu Tsukada, Hideya Ochiai, Hiroshi Esaki

Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2020), Jun. 2020 (Japanese)

conference

Recursive description capable ontology for audio-visual media

Shin Kato, Takuro Sone, Manabu Tsukada, Hiroshi Esaki

Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2020), Jun. 2020, pp. 1373–1380 (Japanese)

Outstanding Presentation Award

conference

Software Defined Media activity report 2019

SDM consortium member

WIDE Report, Mar. 2020 (Japanese)

techreport

3D audio platform for interactive manupilation and AR visualization

Takashi, Kasuya Manabu, Tsukada Ryosuke, Ando Yasutomo, Shirahama Takuro, Shoji Takuro, Sone Hiroshi, Esaki

IPSJ SIG Digital Contents Creation (DCC), Nov. 2019, pp. 1–8 (Japanese)

conference

Web360²: An Interactive 3D Audio-visual Service with Web

Shin Kato, Tomohiro Ikeda, Mitsuaki Kawamorita, Manabu Tsukada, Hiroshi Esaki

IPSJ SIG Digital Contents Creation (DCC), vol. 2019, no. 14, pp. 1–8, Nov. 2019 (Japanese)

article

An interactive remote VR delivery platform for live music event

Takashi Kasuya, Manabu Tsukada, Ryosuke Ando, Yasutomo Shirahama, Takuro Shoji, Takuro Sone, Hiroshi Esaki

Information Processing Society of Japan Transaction of Digital Contents Creation (DCON), vol. 7, no. 2, pp. 1–14, Aug. 2019 (Japanese)

article

Research of Compact Surround Speakers for 3D Audio

Takuro Shoji, Manabu Tsukada, Akira Kato, Hideki Sunahara

Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2018), Jul. 2019 (Japanese)

Young Researcher Award

conference

SDM360²:An interactive 3D audio-visual service with a free-view-listen point

Manabu Tsukada, Yu Komohara, Takashi Kasuya, Hideaki Nii, Shigeki Takasaka, Keiko Ogawa, Hiroshi Esaki

Information Processing Society of Japan Transaction of Digital Contents Creation (DCON), vol. 6, no. 2, pp. 10–23, Aug. 2018 (Japanese)

article

LiVRation: An interactive audio-visual service by using Virtual Reality

Takashi Kasuya, Manabu Tsukada, Yu Komohara, Shigeki Takasaka, Takuhiro Mizuno, Yoshitaka Nomura, Uta Ueda, Hiroshi Esaki

Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2018), Jun. 2018 (Japanese)

Best Paper Award

conference

Software Defined Media activity report 2017

SDM consortium member

WIDE Report, Mar. 2018 (Japanese)

techreport

Software Defined Media Ontology: A framework for reproduction of audio-visual contents by distributed recording environment

Yu Komohara

Graduate School of Information Science and Technology, The University of Tokyo, 2018 (Japanese)

mastersthesis

Software Defined Media: Flexible Services Platform for Audio-Visual Applications

Manabu Tsukada, Keiko Ogawa, Masahiro Ikeda, Takuro Sone, Kenta Niwa, Shoichiro Saito, Takashi Kasuya, Hideki Sunahara, Hiroshi Esaki

Special issue on Network Technology, Japan Society For Software Science And Technology (JSSST), vol. 34, no. 3, pp. 37–58, Sep. 2017 (Japanese)

article
DOI

SDM360²: An interactive audio-visual service for music event

Manabu Tsukada, Yu Komohara, Hideaki Nii, Takashi Kasuya, Shigeki Takasaka, Keiko Ogawa, Hiroshi Esaki

Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2017), Jun. 2017 (Japanese)

Best Paper Award,Yamashita SIG Research Award

conference

SDM Ontology: Ontology for managing metadata of Software Defined Media

Yu Komohara, Manabu Tsukada, Hiroshi Esaki, Takuro Sone, Masahiro Ikeda, Shigeki, Takasaka, Rei Atarashi, Yoshifumi Atarashi

Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2017), Jun. 2017 (Japanese)

conference

Software Defined Media activity report 2016

SDM consortium member

WIDE Report, Mar. 2017 (Japanese)

techreport

Tangible Sound Object: A Tangible Interface for Object-based Sound Systems

Keiko Ogawa, Manabu Tsukada, Hiroshi Esaki, Hideki Sunahara

Symposium on Multimedia, Distributed, Cooperative and Mobile Systems (DICOMO2016), Jul. 2016 (Japanese)

conference

Software Defined Media activity report 2015

SDM consortium member

WIDE Report, Mar. 2016 (Japanese)

techreport

Software Defined Media activity report 2014

SDM consortium member

WIDE Report, Mar. 2015 (Japanese)

techreport

Projects

Web360square
Web360square
Web360square is an application that plays 360-degree video and object-based 3D sounds interactively on the Web.
SDM Ontology
SDM Ontology
SDMO is an ontology to describe metadata in a hierarchical structure for 3D video/audio media and sensor-based data.

Contact

Address

I-REF building, Graduate School of Information Science and Technology, The University of Tokyo, 1-1-1, Yayoi, Bunkyo-ku, Tokyo, 113-8657 Japan ( Map )