WEBVTT

00:01.070 --> 00:05.510
So, one of the things I
was really pleased to hear

00:05.510 --> 00:08.780
is that you were sharing
all the examples of what

00:08.780 --> 00:10.910
you call not so good designs.

00:10.910 --> 00:14.030
But this is the beauty of
design and it's so free.

00:14.030 --> 00:15.920
And there's so much
freedom in actually

00:15.920 --> 00:19.100
being able to design
because you have the freedom

00:19.100 --> 00:21.680
to iterate again and again
and again and again until you

00:21.680 --> 00:22.860
get it right.

00:22.860 --> 00:25.970
And this is really important
when we're using design

00:25.970 --> 00:28.010
to solve complex problems
because we're not

00:28.010 --> 00:31.350
going to solve it in
the first go for sure.

00:31.350 --> 00:34.580
So just having that freedom
to be able to experiment

00:34.580 --> 00:37.950
and having the freedom to
fail is absolutely crucial.

00:37.950 --> 00:41.300
So I think we move straight
to the next speaker

00:41.300 --> 00:43.280
and then hold on
to your questions

00:43.280 --> 00:46.040
because we will have
plenty of time to ask,

00:46.040 --> 00:49.850
but the next fellow
is Jonathan Zong.

00:49.850 --> 00:52.760
He's a computer scientist
and now on his fifth year

00:52.760 --> 00:56.900
of his PhD over at MIT
CSAIL where he's working

00:56.900 --> 00:59.250
with a Visualization Group.

00:59.250 --> 01:01.490
So not only have
you been recognized

01:01.490 --> 01:04.700
as a MAD Fellow,
completely MAD Fellow,

01:04.700 --> 01:10.640
but also recognized as by the
Soros Fellowship and Forbes 30

01:10.640 --> 01:11.840
under 30.

01:11.840 --> 01:17.990
So he's joined today with
his advisor, which is--

01:17.990 --> 01:19.730
now I have to try
to say your name.

01:19.730 --> 01:23.700
I've been practicing, but
we will see how it goes.

01:23.700 --> 01:30.560
So, his advisor is Professor
Arvind Satyanarayan.

01:30.560 --> 01:32.660
Satyanarayan?

01:32.660 --> 01:34.160
Would you try to say that?

01:34.160 --> 01:38.180
My name is really tough to say,
but I think this one is, too.

01:38.180 --> 01:44.270
But anyway, he is
Associate Professor

01:44.270 --> 01:46.460
at the Visualization--
and runs the-- leads

01:46.460 --> 01:48.590
the Visualization
group at the MIT CSAIL.

01:48.590 --> 01:50.870
And before I say
anything more wrong

01:50.870 --> 01:54.320
please welcome
Jonathan to the stage.

01:54.320 --> 01:57.764
[APPLAUSE]

01:58.670 --> 01:59.170
All right.

01:59.170 --> 02:02.180
Thank you so much for
that introduction, Safa.

02:02.180 --> 02:03.910
As she mentioned,
my name is Jonathan,

02:03.910 --> 02:06.670
and I'm excited to
share my work today

02:06.670 --> 02:09.039
on designing accessible
data visualizations

02:09.039 --> 02:12.100
for blind and low-vision users.

02:12.100 --> 02:15.130
So according to a survey
by the National Federation

02:15.130 --> 02:18.670
of the Blind, 7 million
people in the United States

02:18.670 --> 02:20.800
have a visual disability.

02:20.800 --> 02:26.740
And for many of these people
and for more globally, 98--

02:26.740 --> 02:30.970
sorry, 96.8% of the top 1
million pages on the internet

02:30.970 --> 02:34.840
are simply not accessible to
the screen reader assistive

02:34.840 --> 02:38.380
technology that they
use almost every day.

02:38.380 --> 02:40.840
And if you're not familiar
with how screen readers work,

02:40.840 --> 02:43.480
I'm going to kick it
over to my past self

02:43.480 --> 02:46.100
to explain what they are.

02:46.100 --> 02:48.550
So if you're not familiar
with how screen readers work,

02:48.550 --> 02:50.320
there's essentially
a cursor which

02:50.320 --> 02:53.170
will be represented here
in a second visually using

02:53.170 --> 02:54.580
a black outline.

02:54.580 --> 02:57.040
And the user can use
keyboard shortcuts

02:57.040 --> 02:59.200
to move this cursor
around the page,

02:59.200 --> 03:00.970
and whatever is
selected by the cursor

03:00.970 --> 03:03.320
will be read out
as text-to-speech.

03:03.320 --> 03:06.700
So here, I can demonstrate what
the accessible HTML version

03:06.700 --> 03:09.516
of our paper looks like
to a screen reader.

03:09.516 --> 03:11.050
Visit link, heading level 1.

03:11.050 --> 03:11.145
Hi.

03:11.145 --> 03:12.580
We're the MIT
Visualization Group.

03:12.580 --> 03:13.180
Visit link.

03:13.180 --> 03:13.540
Home.

03:13.540 --> 03:15.130
Middle-dot-link,
middle-dot-link, link link,

03:15.130 --> 03:15.490
link.

03:15.490 --> 03:15.940
Heading level 1.

03:15.940 --> 03:16.660
Which screen reader
experiences were

03:16.660 --> 03:18.040
accessible to visualization?

03:18.040 --> 03:19.292
Link, link, link, link.

03:19.292 --> 03:20.500
Past paper honorable mention.

03:20.500 --> 03:21.400
Article.

03:21.400 --> 03:23.290
You are currently on an article
inside of [INAUDIBLE] content.

03:23.290 --> 03:24.220
Heading level 2, two items.

03:24.220 --> 03:24.850
Abstract.

03:24.850 --> 03:26.410
Current accessibility
Guidelines ask

03:26.410 --> 03:27.868
designers to support
screen readers

03:27.868 --> 03:31.010
by basic non-visual alternatives
like textual description.

03:31.010 --> 03:34.030
So that's the experience
of the internet

03:34.030 --> 03:36.760
that you can expect if
you're using a screen reader.

03:36.760 --> 03:40.660
But often when you're
dealing with image content,

03:40.660 --> 03:43.990
if the authors of the
web page haven't provided

03:43.990 --> 03:47.200
an alternative text
description of the image,

03:47.200 --> 03:51.190
your experience will
look more like this.

03:51.190 --> 03:52.660
Image.

03:52.660 --> 03:54.000
Image.

03:54.000 --> 03:55.360
Image.

03:55.360 --> 03:56.860
Image.

03:56.860 --> 03:58.120
Image.

03:58.120 --> 03:59.440
One face.

03:59.440 --> 04:00.730
Image.

04:00.730 --> 04:02.050
Look other.

04:02.050 --> 04:03.920
Image.

04:03.920 --> 04:04.420
Right.

04:04.420 --> 04:07.780
So in the work that I've been
doing with the Visualization

04:07.780 --> 04:09.490
Group and as a MAD
Fellow this year,

04:09.490 --> 04:11.407
I've been kind of thinking
about this question

04:11.407 --> 04:14.980
of how do we make charts, which
are particular kind of image

04:14.980 --> 04:19.152
content, more accessible
to screen reader users?

04:19.152 --> 04:20.610
And there's a couple
of reasons why

04:20.610 --> 04:24.750
this is a challenging
and interesting design

04:24.750 --> 04:26.880
question because
this is essentially

04:26.880 --> 04:29.850
a question of
cross-sensory translation.

04:29.850 --> 04:32.820
Visualizations and
text descriptions

04:32.820 --> 04:34.380
just work differently.

04:34.380 --> 04:38.520
Charts have these various visual
features that, as a reader,

04:38.520 --> 04:41.070
you can kind of shift your
attention around the chart

04:41.070 --> 04:44.850
look at high-level patterns,
look at individual data points,

04:44.850 --> 04:46.620
and you can do this on the fly.

04:46.620 --> 04:49.050
But if you're using
a screen reader,

04:49.050 --> 04:52.500
the text description imposes
a linear predefined reading

04:52.500 --> 04:54.630
order.

04:54.630 --> 04:59.190
So you're beholden
to the decisions

04:59.190 --> 05:01.410
that the person who
wrote the text made

05:01.410 --> 05:06.050
about what information
was important to include.

05:06.050 --> 05:09.880
And so if we walk through an
example of what current best

05:09.880 --> 05:12.190
practices for making
charts accessible

05:12.190 --> 05:14.320
look like using this
example election

05:14.320 --> 05:18.710
map from that video earlier,
there's two main approaches.

05:18.710 --> 05:21.050
The first is alt text.

05:21.050 --> 05:26.490
So if someone were to provide
an alternate textual description

05:26.490 --> 05:29.010
of that map, it might look
like something like this.

05:29.010 --> 05:31.860
A map of 2020 US Presidential
Election results.

05:31.860 --> 05:34.780
Joe Biden has 227
electoral votes,

05:34.780 --> 05:40.610
Donald Trump has 213 electoral
votes, 98 votes remain.

05:40.610 --> 05:44.840
So alt text has some strengths
and some limitations.

05:44.840 --> 05:46.700
One great thing
about alt text is

05:46.700 --> 05:49.580
that it gives you a high-level
overview of the data

05:49.580 --> 05:51.257
pretty efficiently.

05:51.257 --> 05:53.090
After listening to that
couple of sentences,

05:53.090 --> 05:56.930
I know what the general
takeaway of the map is.

05:56.930 --> 05:58.520
But on the flip
side, there's no way

05:58.520 --> 06:01.250
to do self-guided
data exploration.

06:01.250 --> 06:04.370
So if I know the amount of
votes that Joe Biden has

06:04.370 --> 06:06.740
but I want to know
how many of those

06:06.740 --> 06:09.680
votes came from each state,
if that information is not

06:09.680 --> 06:14.850
included in the sentence, then
there's no way to find out.

06:14.850 --> 06:17.220
The other way that
people make charts

06:17.220 --> 06:20.370
accessible on the
internet is by including

06:20.370 --> 06:22.320
a data table of
the underlying data

06:22.320 --> 06:25.160
that the chart is representing.

06:25.160 --> 06:28.340
And data tables address
this limitation of alt text.

06:28.340 --> 06:30.260
They enable detailed
exploration that you

06:30.260 --> 06:32.150
can do in a self-guided manner.

06:32.150 --> 06:34.670
You can use your screen
reader to move around

06:34.670 --> 06:38.450
the different parts of
the table as you please.

06:38.450 --> 06:41.032
But with data tables, it's
very tedious and time-consuming

06:41.032 --> 06:43.490
to build a mental picture of
what's going on with the data.

06:43.490 --> 06:46.710
If you're reading individual
data rows one by one,

06:46.710 --> 06:49.760
there's a lot that you have to
remember in order to understand

06:49.760 --> 06:52.380
higher-level patterns.

06:52.380 --> 06:56.820
So to address the limitations
of these existing practices,

06:56.820 --> 06:59.760
our group at MIT Vis
has been developing

06:59.760 --> 07:02.670
Olli, which is an
open source JavaScript

07:02.670 --> 07:06.450
library for converting data
visualizations on the internet

07:06.450 --> 07:09.540
into keyboard-navigable
textual descriptions

07:09.540 --> 07:12.330
that both allow you to
get high-level overviews

07:12.330 --> 07:15.630
in the form of text and
also navigate around

07:15.630 --> 07:18.550
varying levels of detail and
do self-guided exploration.

07:18.550 --> 07:20.910
And this is something that's
open source and people can

07:20.910 --> 07:23.510
use it today.

07:23.510 --> 07:26.710
So here's a demo video
of how Olli works.

07:26.710 --> 07:30.160
And pay attention
to how we are moving

07:30.160 --> 07:33.995
between text descriptions
at varying levels of detail.

07:33.995 --> 07:36.370
So here on the screen, we have
a scatterplot showing data

07:36.370 --> 07:38.440
about different
species of penguins.

07:38.440 --> 07:41.150
And we have the
corresponding Olli output,

07:41.150 --> 07:43.030
which is a
keyboard-navigable text

07:43.030 --> 07:46.820
structure with descriptions
at different levels of detail.

07:46.820 --> 07:48.460
So let's say I'm
using a screen reader

07:48.460 --> 07:52.690
and I come across this chart
while navigating a web page.

07:52.690 --> 07:56.140
If I read out the top
level Olli description,

07:56.140 --> 07:59.050
I'll get some information about
what kind of chart this is.

07:59.050 --> 08:00.870
A scatter plot showing body mass
in flipper limbs of penguins.

08:00.870 --> 08:03.100
A scatter plot with axes flipper
limb, MM, and body mass, G,

08:03.100 --> 08:03.670
dog.

08:03.670 --> 08:05.920
Now, if I want to explore
this chart further,

08:05.920 --> 08:09.310
I can press the down arrow key
to go down a level of detail

08:09.310 --> 08:10.480
in the Olli structure.

08:10.480 --> 08:12.640
And on that next
level, I'll receive

08:12.640 --> 08:16.187
descriptions of the different
axes and legends of the chart.

08:16.187 --> 08:18.520
X-axis title, flipper limb,
MM, for a quantitative scale

08:18.520 --> 08:21.010
with values from 170 to 230.

08:21.010 --> 08:22.570
And now that I'm
on this level, I

08:22.570 --> 08:24.250
can press the left
and right arrow keys

08:24.250 --> 08:27.633
to go through the different
descriptions on this level.

08:27.633 --> 08:29.800
Y-axis title, body mass,
G, for a quantitative scale

08:29.800 --> 08:31.120
with values from 3,000 to 6,000.

08:31.120 --> 08:32.289
Legend title, species
for color with three

08:32.289 --> 08:33.730
values starting with [INAUDIBLE]
and ending with [INAUDIBLE]..

08:33.730 --> 08:37.360
Now let's say, I want to explore
the chart along the x-axis.

08:37.360 --> 08:39.400
I can navigate back
to the x-axis node

08:39.400 --> 08:41.710
and press down again
to get information

08:41.710 --> 08:44.490
about the intervals
along the x-axis.

08:44.490 --> 08:46.240
Y-axis title-- x-axis
title, flipper limb.

08:46.240 --> 08:48.240
And now, pressing the
left and right arrow keys,

08:48.240 --> 08:50.290
I can move along the
different intervals.

08:50.290 --> 08:51.560
170 to 180.

08:51.560 --> 08:52.060
8 values.

08:52.060 --> 08:53.385
180 to 190.

08:53.385 --> 08:54.130
69 Values.

08:54.130 --> 08:55.120
190 to 200.

08:55.120 --> 08:56.350
113 values.

08:56.350 --> 08:58.870
And by moving along
these intervals

08:58.870 --> 09:01.810
and getting information about
how much data is contained

09:01.810 --> 09:03.970
within each interval,
I'm kind of building

09:03.970 --> 09:07.820
an understanding of how the data
is distributed along this axis.

09:07.820 --> 09:10.960
Now let's say I'm interested
in that interval between 170

09:10.960 --> 09:13.510
and 180, I can go
back to that node

09:13.510 --> 09:16.300
and press the down arrow
again to access a table

09:16.300 --> 09:18.940
of the individual data points.

09:18.940 --> 09:19.885
170 to 180.

09:19.885 --> 09:20.830
8 value.

09:20.830 --> 09:21.580
Table with 8 rows.

09:21.580 --> 09:22.080
Table--

09:22.080 --> 09:24.940
And from here, I can use built
in screen reader keyboard

09:24.940 --> 09:28.745
commands to navigate this table
and read out individual data

09:28.745 --> 09:29.245
points.

09:31.840 --> 09:34.830
So how is Olli different
from prior best practices?

09:34.830 --> 09:37.290
We kind of prioritize
these design goals

09:37.290 --> 09:41.700
of enabling open-ended,
self-guided exploration where

09:41.700 --> 09:44.520
screen reader users can
control their position

09:44.520 --> 09:48.000
within this hierarchy
at their own pace

09:48.000 --> 09:51.220
and also providing descriptions
at varied levels of detail.

09:51.220 --> 09:54.900
So if a high level description
isn't enough information

09:54.900 --> 09:56.610
for you, you can kind
of dive in deeper

09:56.610 --> 10:01.060
and get more specific
about the data.

10:01.060 --> 10:03.750
Now to kind of preview
what's next in this work,

10:03.750 --> 10:08.400
and this is work that I've
been doing for this past spring

10:08.400 --> 10:11.660
that I haven't
shown publicly yet,

10:11.660 --> 10:16.500
we been thinking about this
question of multisensory data

10:16.500 --> 10:17.347
representations.

10:17.347 --> 10:18.930
So currently,
accessibility approaches

10:18.930 --> 10:20.580
are very vision centric.

10:20.580 --> 10:24.450
Here, we started with a
chart and converted it

10:24.450 --> 10:25.552
into a text description.

10:25.552 --> 10:27.510
And we're still thinking
about these two things

10:27.510 --> 10:31.305
as very independent entities.

10:31.305 --> 10:33.930
But what if we think about using
all these different modalities

10:33.930 --> 10:36.900
together in one system?

10:36.900 --> 10:39.452
So here's a demo of
a version of Olli

10:39.452 --> 10:41.660
that's kind of more integrated
with the visualization

10:41.660 --> 10:45.020
so that blind data analysts
and their sighted collaborators

10:45.020 --> 10:47.570
can have a common
point of reference.

10:47.570 --> 10:50.720
And as I navigate
the structure here,

10:50.720 --> 10:52.970
you'll see that
the visualization

10:52.970 --> 10:56.125
updates to show where the
screen reader is focused.

10:56.125 --> 10:57.500
A multi series
line chart showing

10:57.500 --> 10:59.210
stock prices of five
tech companies over time.

10:59.210 --> 10:59.660
One of five.

10:59.660 --> 11:00.800
Line title, MSFT with axes.

11:00.800 --> 11:01.250
Two of five.

11:01.250 --> 11:02.810
Line title, AMZN with
axes date and price.

11:02.810 --> 11:03.320
Three of five.

11:03.320 --> 11:04.790
Line title, IBM with
axes four of five.

11:04.790 --> 11:06.470
Line title, [INAUDIBLE] with
axes date-- five of five.

11:06.470 --> 11:08.220
Line title, AAPL with
axes date and price.

11:10.580 --> 11:13.790
And then we can integrate other
forms of sensory modalities

11:13.790 --> 11:17.270
with their own pros and cons,
like data sonification, which

11:17.270 --> 11:20.990
is very good for giving a
gist of large amounts of data

11:20.990 --> 11:23.150
very quickly but
can't really tell you

11:23.150 --> 11:25.070
about individual data values.

11:25.070 --> 11:27.470
[DIGITAL TONES]

11:33.710 --> 11:36.160
Also that's just
fun to listen to.

11:36.160 --> 11:41.020
And we can't escape
talking about ChatGPT

11:41.020 --> 11:45.070
when we're talking
about textual content.

11:45.070 --> 11:49.090
We've also been kind of
exploring methods where

11:49.090 --> 11:56.200
we query GPT to offer us kind of
additional analysis of the data

11:56.200 --> 11:58.370
that's kind of situated
in the hierarchy.

11:58.370 --> 12:00.488
So I'll give an
example of that here.

12:00.488 --> 12:01.030
Four or five.

12:01.030 --> 12:01.730
Three of five.

12:01.730 --> 12:02.230
Two of five.

12:02.230 --> 12:03.130
Line title, AMZN with axes.

12:03.130 --> 12:03.760
One of two.

12:03.760 --> 12:04.660
X-axis title, date with values.

12:04.660 --> 12:06.160
One of eight, from 1998 to 2000.

12:06.160 --> 12:06.700
Two values.

12:06.700 --> 12:07.660
Amazon, AMZN.

12:07.660 --> 12:09.327
Started the year 2000
with a stock price

12:09.327 --> 12:11.890
of $64.56, which rose to
$68.87 by February 1st,

12:11.890 --> 12:13.300
reflecting the company's growth
and potential for success

12:13.300 --> 12:14.260
in the tech industry.

12:14.260 --> 12:15.700
However, the .com bubble
burst in March 2000 causing

12:15.700 --> 12:18.283
a significant decline in stock
prices for many tech companies,

12:18.283 --> 12:19.190
including Amazon.

12:19.190 --> 12:21.190
And so what I find really
interesting about that

12:21.190 --> 12:26.500
is that this information
can incorporate not just

12:26.500 --> 12:29.290
information that's in
the data, but information

12:29.290 --> 12:32.050
that accounts for things
like current events

12:32.050 --> 12:33.730
and social context.

12:33.730 --> 12:36.730
And that can just
provide another avenue

12:36.730 --> 12:38.630
for helping people
through analysis.

12:38.630 --> 12:43.300
So I've highlighted here a
couple of interesting outputs

12:43.300 --> 12:49.720
that are situated in that Amazon
section of the Olli hierarchy.

12:49.720 --> 12:52.510
So the model is
providing information

12:52.510 --> 12:55.600
about things like
product launches

12:55.600 --> 12:59.260
and entering into new markets to
kind of contextualize the data

12:59.260 --> 13:01.870
that you're looking at.

13:01.870 --> 13:04.930
And it kind of
knows about events

13:04.930 --> 13:07.390
like the global
financial crisis in 2008

13:07.390 --> 13:11.650
and offers that as a
hypothesis for why your data is

13:11.650 --> 13:12.730
the way that it is.

13:12.730 --> 13:17.650
And doing this situated
in the Olli structure

13:17.650 --> 13:20.740
means that rather than
kind of relying on it

13:20.740 --> 13:22.990
to summarize your
entire data set,

13:22.990 --> 13:27.850
you can use it to understand
targeted subsets of your data

13:27.850 --> 13:30.580
and also verify its
claims against the data

13:30.580 --> 13:32.590
that you already have on hand.

13:32.590 --> 13:36.790
So I just want to thank my
enormous list of collaborators.

13:36.790 --> 13:38.380
A special shout out
to Daniel Hajas,

13:38.380 --> 13:42.250
who's a blind researcher
based at the Global Disability

13:42.250 --> 13:46.000
Innovation Hub in London who's
been an important co-design

13:46.000 --> 13:48.130
and research partner.

13:48.130 --> 13:51.520
And also my two current
masters and undergrad advisees,

13:51.520 --> 13:56.500
Shirley Jones and
Isabella Pedraza Pineros.

13:56.500 --> 13:57.680
Thank you so much.

13:57.680 --> 13:59.740
And I look forward to
the conversations today.

13:59.740 --> 14:02.150
[APPLAUSE]

14:06.330 --> 14:06.830
Great.

14:06.830 --> 14:08.610
That was really awesome.

14:08.610 --> 14:10.490
Even though I've seen
the work sort of day

14:10.490 --> 14:12.530
to day, week to
week, it's just nice

14:12.530 --> 14:14.270
to see it sort of
come together and hear

14:14.270 --> 14:17.720
you tell the story of it
versus the story that I tell.

14:17.720 --> 14:21.770
And I was just thinking
about particularly the way

14:21.770 --> 14:25.160
you were introduced as
a computer scientist.

14:25.160 --> 14:29.690
And I'm curious if I can prompt
you to sort of reflect on--

14:29.690 --> 14:32.120
I think the computer
science bit of your identity

14:32.120 --> 14:36.320
is only part of what makes you
such an effective designer.

14:36.320 --> 14:38.360
And I'm curious
about the path that

14:38.360 --> 14:40.670
led you to words
working on accessibility

14:40.670 --> 14:44.390
and how maybe it's
prompting you to draw

14:44.390 --> 14:47.400
on other parts of your
sort of skill set.

14:47.400 --> 14:47.900
Yeah.

14:47.900 --> 14:49.340
Thank you, Arvind.

14:49.340 --> 14:51.990
I guess coming out of undergrad,
I did undergrad at Princeton,

14:51.990 --> 14:54.140
where I studied computer
science and visual arts

14:54.140 --> 14:55.910
with a focus in graphic design.

14:55.910 --> 14:58.880
And I think I kind of stumbled
into data visualization

14:58.880 --> 15:01.430
just because it seemed
like a natural intersection

15:01.430 --> 15:03.290
of those two things.

15:03.290 --> 15:05.173
And within computer science--

15:05.173 --> 15:06.590
like, my undergrad
thesis was kind

15:06.590 --> 15:10.130
of around the ethics
of data collection

15:10.130 --> 15:13.070
and designing systems to
kind of support people

15:13.070 --> 15:17.490
in saying no to researchers
who want to collect their data.

15:17.490 --> 15:21.740
And I think there's kind
of a few through lines

15:21.740 --> 15:24.560
from that work that I've
tried to kind of bring

15:24.560 --> 15:26.220
into the accessibility work.

15:26.220 --> 15:30.740
One is the idea of
designing systems

15:30.740 --> 15:33.920
that support people's
autonomy and agency

15:33.920 --> 15:36.560
and making software
flexible enough

15:36.560 --> 15:41.362
to allow people to express their
preferences and their values.

15:41.362 --> 15:43.820
And I think another is just
the idea of design as something

15:43.820 --> 15:49.010
that can help us answer
or at least make progress

15:49.010 --> 15:53.510
on really tough kind of
social and ethical questions.

15:53.510 --> 15:56.540
In the ethics work,
that kind of looks

15:56.540 --> 16:00.560
like drawing from ideas
in feminist philosophy

16:00.560 --> 16:03.320
to kind of understand
how we should interpret

16:03.320 --> 16:06.050
the way that users interact
with a system that's

16:06.050 --> 16:10.430
designed around them, kind of
saying no to powerful actors.

16:10.430 --> 16:13.280
I think in the
accessibility work,

16:13.280 --> 16:17.480
it's kind of thinking
about broader conversations

16:17.480 --> 16:21.290
in disability studies about
how there's no such thing

16:21.290 --> 16:25.370
as one-size-fits-all solutions
and kind of really taking that

16:25.370 --> 16:28.070
to heart when we
design software.

16:28.070 --> 16:31.430
Oh, I love it because you set me
up for my next question, which

16:31.430 --> 16:35.130
was, so riffing on
that one size fits all,

16:35.130 --> 16:37.340
I was kind of curious.

16:37.340 --> 16:41.510
There's this notion
in accessibility,

16:41.510 --> 16:44.060
which I know you're
familiar with,

16:44.060 --> 16:47.390
but called a curb cut
effect where something that

16:47.390 --> 16:49.910
was designed for
accessibility purposes

16:49.910 --> 16:52.850
ends up having sort
of a larger impact

16:52.850 --> 16:54.630
and benefit for everyone.

16:54.630 --> 16:56.300
And I'm curious
whether you're seeing

16:56.300 --> 16:59.340
some of that emerge
in some of your work,

16:59.340 --> 17:01.730
whether it's in the specific
things you showed us today

17:01.730 --> 17:04.670
or in some of your thinking
around the ethical implications

17:04.670 --> 17:08.490
of this user agency and
autonomy and things like that.

17:08.490 --> 17:09.230
Absolutely.

17:09.230 --> 17:10.369
Yeah.

17:10.369 --> 17:14.569
One thing that we've discussed
a lot amongst our collaborators

17:14.569 --> 17:20.537
is the idea that for
accessible representations,

17:20.537 --> 17:22.079
like textual
representations of data,

17:22.079 --> 17:24.410
why even refer to
the visualization?

17:24.410 --> 17:27.800
Why do like blind users
need to know that there's

17:27.800 --> 17:29.450
an x- and a y-axis?

17:29.450 --> 17:34.700
And I think part of that is just
that we're all kind of situated

17:34.700 --> 17:38.210
in our social and
relational contexts.

17:38.210 --> 17:40.667
Blind folks have
sighted friends,

17:40.667 --> 17:42.500
and they want to talk
to them about the data

17:42.500 --> 17:43.950
that they're looking at.

17:43.950 --> 17:47.300
And so it's important to create
those common reference points

17:47.300 --> 17:50.120
even as we're kind
of translating

17:50.120 --> 17:53.280
the modality that people access
this information through.

17:53.280 --> 17:55.880
And so with the work that
I was showing at the end,

17:55.880 --> 17:59.120
we're really thinking about
facilitating collaboration

17:59.120 --> 18:04.850
between blind and sighted
users in a way that

18:04.850 --> 18:05.780
benefits everybody.

18:05.780 --> 18:07.940
Because having access to
those multiple modalities

18:07.940 --> 18:13.190
just gives you more options
for understanding and analyzing

18:13.190 --> 18:14.030
your data.

18:14.030 --> 18:16.040
And something that
we've talked about a lot

18:16.040 --> 18:20.900
too is there's kind
of increasing momentum

18:20.900 --> 18:23.780
around accessibility in
data visualization right now

18:23.780 --> 18:25.940
and kind of increasing
recognition that this

18:25.940 --> 18:27.270
is an important issue.

18:27.270 --> 18:30.260
And I think part of what we're
trying to show with our work

18:30.260 --> 18:34.010
is that it's both
important because it's

18:34.010 --> 18:37.010
socially important,
but it's also just

18:37.010 --> 18:39.890
like a very interesting
and intellectually rich

18:39.890 --> 18:40.575
area for design.

18:40.575 --> 18:42.200
And there's all these
kind of questions

18:42.200 --> 18:47.330
about what are the different
kind of relative strengths

18:47.330 --> 18:48.980
of these different
sensory modalities

18:48.980 --> 18:52.340
that will then kind of feed
back into creating better data

18:52.340 --> 18:53.900
visualizations?

18:53.900 --> 18:56.360
Yeah, I really love that
point that it's not only

18:56.360 --> 18:59.090
sort of the morally
right thing to do,

18:59.090 --> 19:02.540
but it's just intellectually
deep and interesting.

19:02.540 --> 19:04.710
And these two things don't
have to sort of be mutual

19:04.710 --> 19:06.460
the exclusive.

19:06.460 --> 19:09.930
And so I was curious
about what that

19:09.930 --> 19:16.140
means in the context of that
really sort of provocative demo

19:16.140 --> 19:22.105
that we ended on of the GPT
doing the data analysis.

19:22.105 --> 19:24.480
I don't even know if that's
the right way to describe it.

19:24.480 --> 19:26.880
Doing the data analysis
for us or doing

19:26.880 --> 19:28.900
the interpretation for us.

19:28.900 --> 19:30.330
And so I'm curious for--

19:30.330 --> 19:33.750
I know that it's a
very early prototype,

19:33.750 --> 19:37.740
but I'm curious already what
are some of your hunches

19:37.740 --> 19:42.030
and what the balance we might
want to achieve there might be?

19:42.030 --> 19:42.990
Absolutely.

19:42.990 --> 19:46.290
I went back and forth a lot
on the title for that slide

19:46.290 --> 19:48.930
and how to describe
what was happening.

19:48.930 --> 19:53.280
I think something that's come
up a lot in our conversations

19:53.280 --> 19:58.320
with blind and low vision users
is just the idea of agency

19:58.320 --> 20:02.160
and who is in charge of
doing the interpreting.

20:02.160 --> 20:05.160
When one of my
colleagues, Alan, kind of

20:05.160 --> 20:07.980
tested a lot of different
kinds of textual descriptions

20:07.980 --> 20:13.340
with folks, we realized
that for sighted people,

20:13.340 --> 20:15.480
people really like
descriptions that

20:15.480 --> 20:18.060
do a lot of
interpretation analysis

20:18.060 --> 20:19.770
and give you the
high level takeaway.

20:19.770 --> 20:22.080
Because we can look
at the chart and do

20:22.080 --> 20:27.450
the kind of low level
numbers, accessing stuff,

20:27.450 --> 20:29.250
relatively quickly.

20:29.250 --> 20:31.410
But for blind and
low vision users,

20:31.410 --> 20:33.210
they actually didn't
like those kind

20:33.210 --> 20:36.750
of too interpret-y
descriptions because they

20:36.750 --> 20:38.850
felt like it was kind of
taking away their ability

20:38.850 --> 20:42.330
to draw their own conclusions
first by engaging directly

20:42.330 --> 20:43.660
with the data themselves.

20:43.660 --> 20:48.720
So with something like
using GPT to offer

20:48.720 --> 20:52.500
these additional
descriptions, I really

20:52.500 --> 20:55.440
think that we should be
thinking of these more as kind

20:55.440 --> 20:58.260
of hypothesis
generation exercises

20:58.260 --> 21:01.900
rather than asking
for the answer.

21:01.900 --> 21:04.680
And part of that is, like
John and Maria are saying,

21:04.680 --> 21:09.660
like GPT gets it
wrong more frequently

21:09.660 --> 21:11.580
than we would like it to.

21:11.580 --> 21:13.080
And there's other
work in our lab

21:13.080 --> 21:17.040
kind of thinking about the
ways in which it gets it wrong.

21:17.040 --> 21:23.380
But yeah, I was trying to
articulate in the demo,

21:23.380 --> 21:28.630
situating that additional
information within the data

21:28.630 --> 21:32.680
structure where you can
hear that description

21:32.680 --> 21:36.190
but then go back and do the
analysis yourself and decide

21:36.190 --> 21:40.120
whether or not you agree that
the global financial crisis

21:40.120 --> 21:42.910
actually caused
this dip in the data

21:42.910 --> 21:46.430
is powerful to kind of
leave in the user's hands.

21:46.430 --> 21:46.930
Great.

21:46.930 --> 21:48.670
And I think we're out of time.

21:48.670 --> 21:48.940
Yeah, I think
you're out of time.

21:48.940 --> 21:49.390
Awesome.

21:49.390 --> 21:50.348
Thanks a lot, Jonathan.

21:50.348 --> 21:51.430
This is fantastic.

21:51.430 --> 21:52.870
Thank you so much.

21:52.870 --> 21:54.460
Thank you so much.

21:54.460 --> 21:57.880
And this is a fantastic example
of how talent can transcend

21:57.880 --> 22:01.480
disciplines in what you can do.

22:01.480 --> 22:05.360
Designer turned computer
scientist, or vise versa,

22:05.360 --> 22:07.500
and now solution designer.