Crowdsourcing: Amazon Mechanical Turk
§ A web services API that allows developers to easily integrate human intelligence directly into their processing What can I do for you?
showing fashionable high heeled shoes showing fashionable handbags
Cooking & Health (30) Food (13) barbecue making French fries making sandwich roasting turkey making sushi making salad making pizza making cake making hotdog making cookies making ice cream making Chinese dumplings making egg tarts Drinks (5) making coffee making tea making juice making milk tea making mixed drinks Health(12) Dumbbell workout Barbell workout punching bag workout push ups pull ups sit ups rope skipping treadmill Hula hoop jogging yoga Tai Chi Chuan Leisure sports(7) roller skating fishing boating golfing bowling hiking horse riding
§ On average, 3 tags per consumer video on YouTube vs. 9 tags each YouTube video has
…
Y.-G. Jiang et al., Consumer Video Understanding: A Benchmark Database and An Evaluation of Human and Machine Performance, ACM ICMR 2011.
$ 0.02
9
Fudan-Columbia Video Dataset (FCVID)
• The largest public benchmark (239 categories) of Internet videos with manual annotations • Covering many categories organized in a hierarchical structure • Released in Feb. 2015! /FCVID/
All categories (254) Beauty & fashion (10) Beauty(6) making up make lipstick eye makeup hair style design nail art design face massage tattooing fashion(4)
DIY (21)
singing on stage
guitar performance piano performance violin performance accordion performance cello performance flute performance trumpet performance saxophone performance harmonica performance drumming
Wedding Dance
Beach
Ice Skating
Cat
Birthday Celebration
Music Performance
Playground
6
CCV Snapshot
• # videos: 9,317
– (210 hrs in total)
• video genre
– unedited consumer videos
• video source
– Internet:
• average length
– 80 seconds
• # defined categories
– 20
• annotation method
– Amazon Mechanical Turk
CCV: A Consumer Video Benchmark
• Original unedited videos captured by ordinary consumers
§ Interesting and very diverse contents § Very weakly indexed
wedding ceremony wedding recepNon biking graduaNon baseball birthday soccer playground bird wedding dance basketball beach ice skaNng cat parade skiing swimming dog non-­‐music perf. music perf. 0 100 200 300 400 500 600 700 800 7
海量视频的内容分析与识别
姜育刚
复旦大学 计算机科学技术学院
视频大数据
2
内容提要
• ⺴网网络消费视频内容识别
– 数据集构建 – 视频特征设计 – 深度视频学习 – 其他近期工工作
• 监控视Байду номын сангаас分析系统
– 场馆监控
3
An overview of existing datasets
Leisure & Tricks (22) Common leisure activities(11) flying kites bumper cars kicking shuttlecock playing chess playing bridge snowball fight making a snowman arm wrestling playing with Nun Chucks
Total number of categories: 239 Higher level groups: 11 Finer level category groups: 32
All categories (254) Sports (46) Sports Amateur (14) baseball basketball soccer biking ice skating swimming skiing American football tennis table tennis badminton billiard frisbee shooting Sports Professional (20) baseball basketball soccer biking swimming skiing American football tennis sports track boxing marathon Rhythmic gymnastics taekwondo archery fencing Car racing rowing sumo wrestling diving shooting Extreme Sports (8) rock climbing skateboarding surfing skydiving bungee jumping rafting parkour kitesurfing Sports for the disabled (4) wheelchair basketball wheelchair tennis wheelchair race wheelchair soccer
Dataset Kodak MCG-­‐WEBV CCV UCF-­‐101 THUMOS-­‐2014 MED-­‐2014 Sports-­‐1M FCVID # Videos 1,358 234,414 9,317 13,320 18,394 ≈28,000 1M 91,223 …… # Classes 25 15 20 101 101 20 487 239 Year 2007 2009 2011 2012 2014 2014 2014 2015 Manually Labeled ? ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓
making rings making earrings making bracelets making festival cards making clothes(sewing) making a paper plane making paper flowers knitting assembling a computer assembling a bike
singing in ktv beatbox chorus
rock band performance chamber music
Total number of categories: 239 Higher level groups: 11 Finer level category groups: 32
musical performance without instruments (4)
Music (17)
solo musical performance with instruments (10) group musical performance with instruments (3) symphony orchestra performance