good afternoon it’s a totally wonderful great pleasure for me to be introducing Sarah to you Sarah well have been to students in my time at Berkman who became co-teachers with me the first one Johnny the second is cinema Sarah was my research assistant back aways when teaching internet and society just at the time when we were conceptualizing the idea of Socratic dialogue structure foundation to be scaled as a way of building democratic values into the net and Sarah undertook to teach the Internet and Society with me at the same time she was doing this she was also studying with Catherine MacKinnon in what as far as I could tell from all I could absorb from Sarah about what that was about was a wonderful class in which Sarah opened up new avenues of radical feminism and I listened and felt that she taught me and then saw her teach my students if there’s anyone that I would like to see back in our midst as a change agent it is Sarah so she speaks to you today about the Internet of garbage abuse it’s abuse but she comes out of an understanding of it that isn’t limited to the net it’s just expressed on the Ness and her attention is how to draw it back so with great pleasure I introduced you Sarah John Thank You charlie and thank you everyone for having me here thank you and thank you for having me here is that good all right so road map I’ll first do a little bit of background about the topic and then a bit about myself and then I will be talking about the theory of garbage so really brief background online harassment exists it actually exists it ranges from mean tweets to people calling the police and getting them to send a SWAT team to your house not every instance of online harassment rises to a level of interpersonal violence that requires state intervention but sometimes it does nonetheless it exists in this big mixed-up bucket of all kinds of things and it’s very staggering it’s very pervasive it’s a huge problem on the Internet and furthermore it’s gendered the u.s.National violence against women survey says that 60% of cyber stalking victims are women but the National Center for victims of crime say that’s more like 70% working to halt online abuse looked at 3,000 reports of cyber harassment between 2000 2011 and found that seventy two point five percent were female the Bureau of Justice Statistics says seventy four percent of individual stocks online or offline are female the NYPD computer investigation and Technology Unit the majority of their cyber harassment victims are female and I think this sort of I think this great study in 2006 from the University of Maryland really illustrates what exactly is going on here researchers put silent BOTS like these are just completely silent user accounts on IRC and some of them have very female gender names others have male gender names you know sir like Alice and Bob the bots with the female names received about 25 times more malicious private messages than the ones with the male names these were silent they weren’t saying anything they were just sitting there they were just on the Internet I am going to be talking a lot about gender today but that has a lot to do with the studies that are available as well as the activists sort of focus on this issue you see a lot of the policy recommendations and a lot of the discourse focused around gender but it’s really important to know that this problem is exacerbated on every single axis of social oppression I would particularly like to see more focus on how it effects race because there is a huge problem there and the studies are largely silent as to it sexual orientation is also huge gender identity is also huge and something that we have like no data on is how this problem affects sex workers on the web because they’re on the web they’re people sex workers are people and they get disproportionately so much more abuse than anyone else who identifies doesn’t publicly identify themselves as a sex worker I think that this is a huge problem with our society and it’s worth looking into and something I say at the Berkman Center for Internet and Society for a good reason this inequality and participation on the Internet has a measurable cost to society 91% of wiki meet Wikipedia editors are male and surprised it’s harder to find articles about women on Wikipedia than it is articles about men there is a measurable impact on what happens when women are harassed and silenced when they feel as though they’re online and in a culture of toxicity that excludes them so like I said earlier harassment is a really big bucket you have I think everyone every woman almost experiences this constant low-level buzz of harassment it’s not that big of a deal but it’s kind of really annoying right it happens it’s everyday or close to everyday it’s just sort of there like what why are these people bugging me it might not be very violent it might not be very upsetting but it’s it’s there and then you have the targeted campaigns and these are the things that make the news these are you know this is like Anita sarkeesian having to cancel a talk because of a bomb threat this is south-by-southwest canceling two panels because of threats this is gamergate which by the way is is actually an example of domestic violence rather than anonymous strangers taking someone down it’s an instance in which an ex-partner decided to go after a woman very violently and that’s something that we shouldn’t forget like this like we like to focus on stranger danger we like to focus on scary anonymous people but really the scariest people among us are the people we know so here’s some tweets at Carolyn creado Perez who was a female journalist in Britain who was wanted to put Jane Austen on British currency so she got tweets like this off and die you worthless piece of crap go kill yourself rape is the last of your worries shut up goes on and two people were actually jailed over this eventually here are some comments posted on anita sarkeesian’s videos I hate ovaries with the brain big enough to post videos fun aside she completely forgot to mention that every guy in video games has these stereotypes – do you see us parading about it about it no honey it’s a video game tits or get the out yeah I can’t wait for the day we get to play ugly feminist hand planet the game that would sell millions of units as you can see not all of this is you know straight up I guess crass or not all of these are threats but it’s not nice to be reading this day in and day out I’m not saying you should get imprisoned for not being nice to people on the internet but this is a bit much there’s a lot going on here so compare this to sort of like I was basically looking at a screen grab right like I took these from a screen grab so this is like one comment after another after another I wasn’t like going out of my way to find the nasty ones so here’s you want to sort of balance out as a palate cleanser I am okay with this and I believe everyone is – case dismissed right great that that was the most positive thing I could find on that on that screen grab so these things were happening actually they were just getting started while I was still at Harvard the discourse was shifting I had come to HLS in a time when the the frame was still very focused around free speech good oppressive governments bad we must protect the internet from these these threats of interference of censorship and suddenly the discourse was changing and we were increasingly seeing these examples of women getting basically threatened off the internet and this wasn’t new but suddenly there was more and more attention to it and I think that has in large part it’s because of Twitter it’s because Twitter is open and it’s because we can see it happen in real time ourselves it is no longer a subjective experience it is open objective and available to the public for everyone to see and it’s horrifying and the discourse started out as this is free speech versus women and this was pretty upsetting to me I was you know coming from a place where I believed in a free and open Internet I did not think that many of the proposed regulations coming out of these episodes were good I didn’t like the idea of people getting sent to prison over tweets even though I looked at the tweets and I they were very upsetting to me and as Charlie mentioned I was here I was I was working with Charlie on the Internet in society class I was also at the journal long gender I was very involved in feminist issues and activism and I felt like I was being told that my position was inconsistent because it’s free speech versus women pick one side no one wants to be on the side that’s home I guess women gets lose so the more I thought about it though there’s a there’s a thing in you know First Amendment jurisprudence known as as the hecklers veto actually and the hecklers veto is sort of this problem where if you yell at someone enough it causes an issue where oh okay I guess it’s now legal to shut down the speech because people are it’s just upsetting everyone way too much it’s like causing it’s fighting words now because it’s causing people to fight against it or now that it’s it’s causing all of this violence to erupt then we gonna have to shut all the stuff down and that’s how I viewed harassment harassment was a hecklers veto it wasn’t the case that harassment was that was the speech that was under threat it was the speech of women that was under threat because women were the ones who are being told to leave these social platforms who were leaving these platforms who were ceasing to speak who were being silenced and when we’re talking about free speech for women I mean we’re just talking about free speech obviously this is all well and good and you may say this is little more than a cute rhetorical trick to justify the suppression of speech and I assure you that it is not I have learned many cute rhetorical tricks from this institution where many of the masters of cute rhetorical tricks reside but there’s actually more to this and I am going to explain it goes back to the history of the internet and some largely today overlook the facts about Internet infrastructure this is what I call the theory of garbage the theory of garbage is very simple the internet is mostly garbage well there’s more to that so what I’m talking about garbage I am I’m talking about things like spam I’m talking about things like malware and I want to include harassment these are just things that we have come to accept aren’t great and there is certainly a contingent and I have to qualify before I even begin going into this look spam is speech malware is speech like code is speech harassment is speech nonetheless this all belongs in a bucket where we know that something has to be done about it it’s doesn’t mean we have to put people in prison it doesn’t mean everything has to be deleted but there’s an issue here it these things make the internet unusable and I started thinking about this actually because I was having dinner with these older attorneys who had been working with internet law for a long time and that oh man remember when people were really upset about server-side filtering the spam filter I’m like I know that’s before my time like what what happened there and they basically spun out the story that I I thought was incredible I will leave out names but basically around this era in the earlier 2000s you had you know people who are typically you know free speech wing of the free speech party like you know these are the the kinds of people who make the ACLU look like a bunch of Europeans all right so these are these are people working in the tech industry who are really really like nothing should be censored on the Internet and when it came to spam however they were just like burn it with fire rain nuclear bombs on spammers this is the worst thing that has ever happened to the Internet and how dare you tell me that spam filters might be bad so spam comes of course from the Monty Python sketch where two people are trying to talk and these Vikings keep interrupting with the spam spam spam glorious spam spam and spam and you see it probably get its name according to fan Brunton from this IRC trick people would do where if like just for you know fun they would type in spam and then hit enter and then hit the up-arrow which just repeats the message up arrow enter up arrow enter up arrow enter and so the entire chatroom would flood with spam spam spam spam spam making it impossible to talk and so you get this name and eventually you’re starting to get a lot of unwanted messages through various system so this exists even before email is really email this exists before the web is white the web this exists on using that this exists on things like CTS us at MIT in 1971 it’s it’s just unwanted mailings and obviously this is a very subjective definition right keep in mind this is very subjective and keep that in mind as we go on to talk about harassment which is obviously a very subjective definition as well and eventually people create anti-spam technology they create filters they start out with very naive filters regular expression filters which are just you know oh well let’s just filter out every single male email that says viagra which by the way there was a woman Italy called name’s Olivia grid’ inna and all of her emails disappeared into a black hole for like a few months because of this and so the regular expressions not great and around early 2000s the e FF put out this position paper on spam filtering and like any good position paper in a time of great outrage and and controversy you can sort of read what the controversy actually is because they’re carefully navigating around it like the language is pretty incredible there they’re basically a look spam is yeah we know spam is bad but hey people are losing moveon.org emails in spam filters and you know moveon.org that’s political speech that’s speech a lot of the moveon.org emails that we’re getting lost in in the spam filters were actually like action alerts like you have to call your Congressman now we have like one hour to call your Congressman about this bill and if that gets even delayed for a bit in a spam filter that’s political speech being silenced that has actual implications on our democracy and what the e FF was recommending at that time was look server-side spam filters so spam filters on the side of the mail server we’re the people themselves don’t have control over how to tweak the spam filter or whatever those are problematic those are bad those are a threat to free speech client-side spam filters good nowadays we all use Gmail and that’s that server-side although it takes into account our own preferences so it’s kind of client-side kind of server side so that distinction is now largely defunct but the issue is still live so spam a speech it’s always been speech actually so fin Brunton says that the first instance of spam was in 1971 on the compatible time sharing service at MIT which was this network and this one engineer or I guess he was assistant min got access to the entire mail like to the entire list or of people and he was like all right I think I really need to send this message out to everyone it was an anti-war message he wanted to tell people that you know it was their moral obligation to oppose the Vietnam War he was mailing out to a bunch of people who were taking DoD funds at the time so he was really good this is this isn’t an irrelevant audience this is a very relevant audience we need to take a moral principle stand and so the very first bad message is there is no way to peace peace is the way and then you see a few decades later sort of as people are discussing can spam and other anti-spam regulations you have the spammers themselves going spam a speech you know it’s what America was built on small business owners have a right to direct marketing but eventually can spam passes and spam actually ends up becoming largely the enterprise of criminal groups in Russia it’s kind of an odd twist but most of the spam that we’re getting over the internet now is basically run by the Mafia because sort of the small business owners who have a right to direct marketing decided not to do this anymore because it was illegal and so now we have what we have now is you know the viagra emails they’re like barely readable they’re sent by BOTS there are botnets there are computers that are controlled by viruses that are there their cycles are getting eaten up by sending out all of these mass mailings and it’s um it’s kind of a terrible situation and in fact spam is a huge infrastructural menace in 2013 there was a Gartner report that said that 69 percent of all email is spam and this was a huge reduction like a few years earlier been 89% and they were like yeah no this great reduction doesn’t have anything to do with anti-spam technology this is actually the spammers are moving on to the social networks because that’s where people are that’s the that’s the right market I mean I’m sure some of it has to do with anti-spam technology but it’s like it’s actually quite dire the email security market is estimated to be about 7 billion dollars American firms and consumers experienced 20 billion dollars of loss annually because of spam and what’s really sad about this the spammers actually only gross about $200 million total for this $20 billion 20 billion dollar loss it’s it’s a it’s not great spam is actually really bad for the internet the internet is at least 69% garbage I can tell you that with some accuracy but at the same time spam filters are still a free-speech issue things are still getting caught in there we don’t think about spam filters that much anymore we just don’t think about spam as a problem anymore because it we all use Gmail and Gmail anti-spam is pretty good right like now and then we’ll get something in our our spam folder that shouldn’t have gone in there and we’re like oh this is the worst like that I was supposed to reply to that like last week and now something is is the is wrong but that happens very infrequently we don’t think about spam anymore we think of spam as a salt problem but it’s not it’s not at all it’s it’s an ongoing free speech issue things are constantly getting stuck in spam Phil I shouldn’t be stuck in spam filters and right now you have this thing actually where Gmail is filtering out mail coming from self hosted servers at this point like they did a tweak recently and so that’s a huge problem too because now it’s basically privileges people who are using services like Gmail over people who are trying to keep the web decentralized and to run their own own self hosted servers because you know various political stances or they just have a lot of free time on their hands like it’s but that’s important like this is an issue that this is about the decentralization of the web turning into the centralization of the web this is about Google having rather undue control over people’s communications to each other and spam definitions in the end are quite subjective right it’s even though we have a sense of what it is just like we have a sense of what harassment is we have a sense that someone typing you at someone else that they don’t know is harassment it is nonetheless like we can’t just put a filter for you and impose that on the Internet in the end harassment is very subjectively defined and spam is also subjectively defined and of course you know there’s the old adage one man’s trash is another man’s treasure and this subjectivity is very problematic like we don’t want to be filtering or blocking or deleting things that that you know another person argues that’s actually good but look we still have a municipal waste system we have to function as a society by taking out some of the garbage like you have now we can upcycle some stuff we can recycle some stuff and some stuff just shouldn’t be thrown away but in order for things to function we have to make sure that email isn’t visibly 69% spam even if it actually a 69% spam so I go through this history because harassment is spam when you’re looking at the constant low buzz the targeted campaigns these news articles about oh she received a flood of tweets and then they will even say how many tweets per hour she was receiving what you’re looking at is a targeted personal distributed denial-of-service and in fact the common sort of advice in these situations is just turn off your phone for a while get off the internet it’s basically social media no longer functions for you you no longer have this avenue of communication your machines have been in a way compromised by too much information because people are screaming at you too much and this is something that clicked for me when I I heard this interview with Mickey Kendall I mean this is something that had been brewing in my head for a bit and I but I think that she she says this like pretty greatly she received receives a lot of abuse she is a black woman and she is public and she’s out there on the Internet which means that she receives tons of abuse and one day she and some other friends decided to switch out their Twitter avatars to white men and what they saw was that the abuse that they were receiving was suddenly diminished like the same and in some cases the same people were interacting with them but suddenly the those interactions were like oh they were they’re reasonable they actually want to talk about the issues instead of just being jerks and she said one of the things that’s really nice is not waking up to c62 comments calling me everything but a child of God these are people who are getting spammed and they’re getting spammed strategically anti this is anti speech online harassment is strategic spamming flaming DDoS anti-spam moderation anti-ddos these are struggles over the forum manifestation and social norms of the Internet itself these are people fighting out what the definition of garbage is because they have differing definitions of garbage and what belongs on the internet and what doesn’t and they’re using these tools to hash that out and for misogynists women are garbage and they have to be driven off the internet because they don’t belong there no more than the spam does so what we’re looking at is that anti-spam technology the code around it is political it’s not neutral in the slightest when it comes down to it the first page of Google search results can only have so many things right so even if you’re not blocking everything you kind of like some of that stuff is getting lost when you’re looking at Facebook algorithms and how they get sorted some of those communications are getting lost and some of them are getting promoted when you look at Twitter blocks and how they work and I bet this is something that a lot of people didn’t think about or a fair number of people did think about a few years ago but they changed how blocks worked and they made it so that people could still follow you and interact with you even after you blocked them and and before you couldn’t do that and there was this huge outcry because they’re like this would exacerbate online online abuse and this is interesting because that’s just a tweak to the code right this is just this is this is a user interface thing changes seasoner interface are not neutral they will hurt some people and help some people sometimes they’re very very political and actually you sort of see this use of code on the rise right now to deal with harassment you have and it’s rising in parallel almost with a history of anti-spam so you have regular expression filtering which by the way is is still terrible right so I actually have a friend who uses a client so that gamergate never appears in her timeline and then but because people never type out gamergate she had to also censor Gigi which means that if I type out something like argh that also gets censored and she never receives it so like so that’s that’s the early side and then we also have client-side blocking mechanism we have things like block together which is a subscribable blockless service we have things like gigi autoblog which automatically calculates whether someone is likely to be a member of gamergate and then puts them on our block list so that you can subscribe to that block list and lock that person you have the block bot you have these basically very early almost vigilante forms of filtering and these actually resemble closely how spam filtering worked in the really early days before email became hugely centralized under under Google so this is I think a really promising place to go towards and where we are right now in the policy discussions is not so much infrastructure and code it’s actually deletion censorship and imprisonment in some cases and one way to think about this is that code is frequently an anti hawk solution you’re setting up the conditions under which people speak to each other this is you are trying to prevent harassment before it happens by making you know better user interface decisions by making sure that your algorithms don’t privilege one group over another or just making it so that people have options when they’re getting harassed deletion is post-hoc it’s after something happens you hide it you sweep it under the blanket and unfortunately deletion is there’s a lot of focus on deletion and it’s filtered not just into the policy debates but into the legal legal debates like when we think of legal solutions to the Internet of garbage we think in terms of deletion and I think that as someone as a scholar of copyright law I think that this has a lot to do with the very privileged place that copyright law has in our society copyright law is the easiest way to get something deleted it’s the easiest way to fight back against someone on the internet because copyright law is overpowered it’s it’s not at all a balanced law and we are looking right now at this new problem at well not quite new but a very different problem a problem that we haven’t quite thought of in the terms that we’re thinking about now we’re looking at it through the lens that the RI double a has created we’re thinking about notice we’re thinking about takedown we’re thinking about going finding people we’re thinking about subpoenaing ISPs and this isn’t useful and I’m going to run through a very unusual Ninth Circuit case to explain why this isn’t very useful otherwise known as how to break copyright law so Garcia V Google in the Ninth Circuit this is Cindy Garcia she was recruited to be in this movie called the desert warrior and it was about as far as she knew it was it was a perfectly nice film it had something to do with the Prophet Mohammed but it’s it’s a fine film later she found out that her parts had all been dubbed over and that hadn’t been released as the innocence of Muslims this did not go well for Cindy Lee Garcia so of course her her name got put out there I her personal information got put out there there was a thought on her these are some of the things that were said to her and these were read out loud in oral arguments when in the on bunkering so there was an initial decision and then that was appealed back up again to the Ninth Circuit so there’s decision Garcia won and then this is Garcia – so this is quoting from the hearing at Garcia – are you mad you dirty I kill you stop the film otherwise I kill you hey you why you make the movie innocence of Muslim delete this movie otherwise I am the Mafia Don I kill whoever have hand in salting my profit Oh enemy of Allah if you are insulting muhammed prophet’s life suffer forever never let you live it freely sore and painful wait for my reply and this was interesting so counsel was reading these quotes into the record at this hearing and it was strange because this is a copyright case and actually almost everyone who’s there they were copyright lawyers they were really interested in sort of the weird copyright angle and including myself and the immediately there are like wires why is she reading these threats out loud like what does this have to do with anything and in fact the the judges agreed immediately you get sort of an interruption that you never want to hear if you are arguing a case in front of the Ninth Circuit Council how do these threats go to the preliminary injunction standard so yeah so Garcia was was a very problematic case so in Garcia one you got a judgment in favor of Cindy Cindy Lee Garcia and it was basically the Ninth Circuit world yeah she actually does have a copyright in her performance in the innocence of Muslims now Lindsay Lohan does not have a copyright in Mean Girls Leo DiCaprio does not have a copyright in Titanic like that’s not how copyright law works you act in a movie and then you get paid and then someone else owns the copyright on the movie and usually there’s like a series of very complicated contracts to govern this but it’s largely assumes that even if you like are making like a home video without any contracts whatsoever the actors are not the ones who have the copyright in this work and so this decision basically broke copyright law it broke the fixation requirement it created a new form of authorship it created a new subject matter for copyright it messed around with the definition of performance in the statute it had a really weird effect on the movie industry potentially like they’re one of the questions I was asked during oral argument was could any person who appeared in the battle scenes of Lord of the Rings claimed rights in the work feasible it broke section 512 which is the notice and takedown section of the DMCA and it also broke CD section 230 which which would have basically otherwise immunized Google from from civil suits and so this is interesting like all of these things happened because basically two out of three judges felt really sorry for Garcia like this is a really bizarre unusual decision it has no place in law it is one of the most bizarre things that I have ever read actually and and it and all for what it was to take this movie off of YouTube and what would that have done like it’s we think in these terms of deletion and it gets us nowhere because deletion isn’t victory it’s not liberation it’s not freedom from fear it’s just deletion and you see this actually in like a rather unfortunately callous comment by one of the judges and the on bond hearing is there anyone in the world who doesn’t know your client is associated with this video oh maybe in a cave someplace and those are the people we worry about but we weren’t able to give Garcia any real measure of safety through this and it means that this focus on deletion is not that productive and obviously I did spend a fair bit of time talking about filtering but I was talking about filtering from the perspective of how can women be on social media and be on social media without getting driven off of social media immediately it’s not deletion is a tool it’s not the end goal deletion is a tool to give people space on the internet and I say deletion but really I would prefer to use the word filtering because you know people can yell at me all they want I just don’t want to have to hear it and I so I think that a more productive direction is for us to shift away from these post hoc reactive measures and look at things like a compassionate user interface so an interface that understands that these things happen and like maybe you know after the fifth time you type something furiously at someone the interface says hey are you sure you want to send that message something that just checks you not necessarily something that silences you just something that checks you restorative justice I think is also a very very promising frontier so Riot Games is a games company that publishes League of Legends which is known for quite a lot of harassment just really uncouth behavior between players and they recently instituted a series of internal reforms it’s an ongoing experiment to try and make things a little more civil inside the game and actually so they set up this thing called the tribunal system where the players would actually get judged by like basically a jury of their peers like it when they got flagged with a report they would get put through the tribunal system and most of the time and the tribunal system is checked by the way by the staff so the staff will almost access I guess in an appellate court or a judge in this case and the the tribunal will say like no your behavior was out of line or no it’s okay that that report was no good and the players who are being judged by other players often it turns out they they didn’t realize that their their behavior was was wrong or it was unacceptable to the community and they will apologize they will even voluntarily put themselves in in restricted chat mode which means that they only get like you know a certain number of chats and so they now have to choose between talking to their teammates or or trash-talking right so it it puts their focus back on the game it regulates their behavior and this is something that they opt into and and I think that this kind of thing is really important and very promising something that I do want to caution you is that it doesn’t necessarily mean that all people who are harasser czar just like somehow misguided or they’re just immature children who need to learn better it could be that these are all just incurable sociopaths but that doesn’t matter they’re trying to do better anyways because now that the eyes of society are on them they want to change they do not want to be perceived as someone worthy of censure and so what I’ve done here in laying out the theory of garbage is actually not that interesting is it’s I’ve basically applied Larry Lessig to online harassment so there’s Larry Lessig’s for forces you’ve got market law norms architecture and so on the law side you have things like the DMCA cyber bullying laws revenge porn laws and and actually social media policies so these are not laws per se but they are becoming the new law what Facebook decides is acceptable on their platform that is a an extra legal determination but it’s one that looks much more like a legal determination than anything else you have norms that’s just moderation just telling people hey that’s not okay that’s the tribunal system it’s people coming and going why would you say that like this is not the place for something like that these are the things that keep people in check who these are the things that make people feel as though just yelling at someone is is not allowed even if you know no one has expressly said that to them maybe and then there’s architecture that’s interface design its algorithmic self-help like these blocking tools this is this is the code this is the code that is never neutral and this is where I’m hoping we get some respite from the barrage of online abuse if we can just design the architecture of the Internet in a compassionate a gala terian way we can get someplace we can get someplace much from here and if anyone figures out the market angle to this just let me know because I never did so when we’re looking at online harassment we can learn from the history of spam and you know it’s just spam speech spam is also garbage and so we carry on it’s a we can go move forward from there we don’t have to stop it but what about my free speech the conversation cannot end their speech is important but we have work to do anyways so in conclusion the Internet is mostly garbage garbage is a speech issue but we still take out the garbage code is never neutral we should have thoughtful design for the marketplace of ideas and that will make us a better future thank you so we have about 15 or 20 minutes for questions Sarah I love it right up until the end and then at the end I lose it it’s like somehow you’re gonna try and create a norm on the open net but it doesn’t work that way when we try and do it in a real world you find that you have to start with an affinity group like this group in this room is we are a group now if we were going to talk about something and do it in a way that was not mutually insulting that drugs up and gets to be gamergate we could approach that problem and in fact when we’ve tried to do this we have used our internet questions to do it by starting for example on the gender issue starting in a group like this mix male and female by anonymously feeding into the questions rule what your fears would be about actually talking with opposite numbers about delicate issues of diversity and then anonymously feed in what your hopes would be if you could do that and then structuring a discussion that’s got a real issue to talk about and setting people to talk and lo and behold with the aspirational view and the fears people are considered as they approach it and can do well with it but if you tried to do that on the open net you just get creamed so my question is as you’re trying to use architecture if you’re going to limit yourself to the architecture of the open net I don’t think you’ve got a winner but if you can somehow combine the architecture of real-world environments with real-world affinity groups and then scale to the net now you could have stopped establish some kind of structure so I think that the open net is actually just not as open as we often say it is in our rhetoric it’s it’s enclaves its enclaves of people it’s it’s clusters of communities it looks more like a cluster of grapes than it does a globe it’s so there’s some places where the internet really sucks and then there’s some places where the internet is really great I think a really hard problem is when those worlds cross over when what is known as a forum raid where people from one of these worlds goes into the other and just disregards the norms there and I think that’s a really hard problem but beyond that I think that there is absolutely there is absolutely a way to set norms on the internet without ever seeing people face-to-face this is you know people make friends on the internet and then they they never see them in real life ever again they do this they form these communities they have they are they are truly people on the internet there are truly citizens and residents of the net and of the web and they speak to each other they form norms and they form societies and I think this is this is absolutely possible there are a lot of hard problems here but you know when you speak to people who do community moderation for instance you can really sort of hear them talking about a craft of basically of setting a level of discourse of this this there’s this emotional care labor here that where you can make people comport themselves in in a way that’s that’s better or worse you can make things worse like very easily and it’s very it’s much harder to keep things civil and useful and also to keep people ongoing and then also to encourage people to speak but also to get that you know that one person who speaks way too much to like pipe down a little bit this is this is work and I think it’s entirely possible to do it just on the internet I guess this is a related question about twenty years ago a Cass Sunstein wrote a book called Republic comm and he was concerned not just with harassment but in fact extremist speech and he talked about how the internet made possible ideological echo chambers that seemed to encourage this kind of extremist speech you know this goes now the reason I bring it up is that you’re throwing every throwing spam malware and harassment into one into the garbage pail together I understand everything’s the same if you ignore all the differences but I I’m not sure I think for Sunstein free speech aside there was a kind of communications disorder that this isolation created and when you look at it from that point of view the remedies are like what you’ve just mentioned but also it gets you away a little bit from this question of whether you’re have to take sides between free speech and harassment because that’s not really what you’re talking about so do you have no faith in the up in the ability to address communications disorders this way when you’re talking about things like storm front right like that’s you know storm friend storm front right people go like it exists and people go to storm front to be white supremacists but then you’re looking at things like for instance enclaves created on Facebook right which is a neutral platform and because of the way say algorithms work or the way people are able to find each other on Facebook you have a communications disorder there and in that case it does sort of become a bit of an infrastructural problem and that people create these echo chambers through the infrastructure that’s supposed to be neutral I’m not entirely sure that this means that people should be hit with like a daily dose of something that goes against their political beliefs but yeah there’s like some of its related some of its not and I one more thing is that when I say harassment belongs in this big bucket I do mean that in the same sense that spam belongs there because when it comes down to it spam is not it’s a meaningless word like we do know what spam is but when it comes down to it spam is just an unwanted mailing and that means nothing I’m probably one of the newer people to this face but I gathered from your presentation that there seems to be a shift from punishing content to regulating users and I was wondering if the the hecklers voice is a direct analog to that and also if inherent in the market is kind of an aversion to regulating users and a preference to punishing content like with the Twitter okay so I think it’s in especially in the u.s.It’s a lot easier to go after the content and that has a lot to do with the way copyright law has has structured the web basically there are avenues to pursue content and people add companies who are supposed to deal with these requests so there’s like there is a a bureaucracy in place for that and in the US were really leery of the idea of actually going after the users not so much in other countries this was a pretty us-centric presentation I’m afraid the concerns are very different in other countries and of course free speech is a very us centric topic especially when you’re talking about it in the US so I’d have to say that that totally differs were where you are I fascinating talk and as the son of a man you used to build sewage treatment plants I appreciate the trash and garbage framework because there’s always trash to take out I’m also somebody who sort of lived the spam Wars as a you know Alton store I’ll install the server-side filter when you pry my cold dead hens from the server side and what a lot of that was I and I eventually did install server-side filters for a university in the area without them having to pry my cold dead fingers because it need to sort of use your frame here what we were doing was victim-blaming well you just can’t manage your mailbox you just need to delete that you need to just not read that and it’s exactly the same you know sort of thing people are saying to women about harassment well just don’t pay attention turnover for whatever out of span came a seven billion dollar email security industry so I think you actually do have your market solution there you just need to find a way of turning this into you know a profit opportunity for somebody and I’m I actually mean that seriously in the sense that why isn’t the seat I mean ask the question why is the scene as a commercial opportunity for you know the various platforms to do this to find a way to actually make money and is that just because women’s experiences are completely devalued I mean if it’s might be that it might be that women earn 70 cents to a dollar it may also be that I mean this is this was a snide thing that I said in my book but if harassment seems very like a very feminine problem then spam seems like a very masculine one it’s like you know viagra penis enlargement mail-order brides so it’s it’s a it’s a problem that that is very in-your-face when you’re a man and harassment is not so much in your face when you’re a woman and I mean guess who is has buys shares in the email security market and who actually works in the email security market and and all of that labor capital from top to bottom it’s all men so I don’t know that’s there’s structural issues there that go way beyond the scope of this talk I’m just going to bounce back on your remark about this presentation being a very you eccentric from my point of view when I’m from a European law background I’m fascinated by the power that the words free speech and censorship have in in the u.s.And in particular how how we all agree that there’s something wrong with hate speech and all this garbage on the internet but we absolutely refuse to touch speech and even have to break copyright in order to to solve these problems and we always do it in roundabout ways instead of trying to address the actual problem that’s with here maybe two examples that go in this direction when you have some stakeholders that try to get their responsibility when it comes to speech they just say that it’s censorship as if it was an argument enough that’s enough itself you can look at google when they try to implement or right to be forgotten it was eccentric as if it was the entirety of the debate also when we were talking about the the negotiation of the ITR in the ITU most of the issues were just about internationally not internet regulation and more about like tell economy connect telecommunications but then the free speech free internet issue sort of rose up because the US didn’t want to move on some of the other positions but everything got swept up into free speech issues and nothing got disgusting young do you think we can perhaps move towards a clear a discussion of free speech issues well we have to avoid the elephant in the room for a long time and so I think that this is just sort of a big difference between how Europeans and how well nor North Americans maybe because Canadians to some extent do you subscribe to but not quite so let’s just say Americans this is just a big difference between how Americans and Europeans approach approach the issue this is just I think that two people outside the US are fixation on free speech appears to be a sort of collective insanity that said I subscribe to the collective insanity I think it’s great I think free speech is great I’m I am not sure if this is just sort of one of those intractable things or if because when it comes down to it right there’s there are some things that are just closer to being inherent goods then you know other things right like we talk about equality we talk about but freedom in general and and to some people actually maybe equality is not that good and why do we care about it so much but it’s but to to other people it’s not even quite worth the debate like that’s like something that it is maybe best left to you know first year philosophy classes I think like I’ve given some thought to this I’m not really sure where I come out on it I think that in in other countries outside of Europe and outside of America those perspective terms I find very interesting as well because you have but you have like a very very different threat model there especially under oppressive regimes so I think that it’s it’s largely a circumstance how you perceive your own government and your own society and in in the states like there are fixation with free speech says something very interesting about our society but that’s that’s neither here nor there there’s a traditional liberal response that the way to deal with bad speech is to drown it out with better speech why doesn’t that work here or does it work here because it just turns into a shouting match and you can’t it’s I framed this as a usability issue I don’t think it worked before the internet either like I mean so I do in practice I do engage encounter speech and I think counter speech is good it’s it’s a tactic it’s certainly a tactic but it can’t be the only tactic like it’s one it’s one tool in this toolbox there is a big toolbox and I don’t think that counter speech was ever just the answer because when it comes down to it we you can only hear so many decibels right and you can only see so many messages at a time they’re like we’ve we’ve always been in a place where not everyone gets to speak and this is a problem that I think or rather everyone gets to speak but not everyone gets to be heard and this is this is just something that we don’t talk about enough I think high so you mentioned this idea of the architecture and the design of the Internet and as a female user of the Internet and developer I feel like psychologically I’ve had to lend myself to a certain way of thinking in order to participate in the design of the infrastructure and I wonder if you think that you know in a world where a lot of this architecture is designed by men if maybe equal representation the technology sector may be a possible solution to designing interfaces or just uses of the internet in a way that adheres to female psychology I also study cities and it’s quite similar there’s a number of studies that explore how the urban environment the built environment is also a product of sort of like masculine psychology and that you know the representation more representation of women in the design sector creates environments that women feel like they can participate in more so I’m curious as to your comments on that um 100 percent more female engineers more female CEOs more female venture capitalists more female board women on boards of directors it’s it goes all the way to the top it’s from the top to the bottom everything is implicitly organized around how men see the world and and not just meant how white men see the world and this is this is a problem this is why so many things suck and and and this this should absolutely be addressed and it should be addressed for more reasons than just oh like we’re not able to expand to this market out because it turns out this product doesn’t understand a user base beyond white men who are not exactly a majority in the world it’s but that’s the start you know like eventually you’re going to tap out your tiny minority user base and you’re going to have to work on a national level on a global level you’re going to have to reach out to everyone because inclusivity isn’t just a social good it’s a it’s financially productive hello I’m Matt it and so I was just curious what you thought about uh the role that a 230 and safe harbor immunity is played and I know you said you didn’t really think about like a market solution but what if we repeal to 30 and then Twitter was like under threat of litigation for inflicting what they are upon the world I mean that would certainly shift market dynamics to 30 good people who hate 230 stupid it’s like you know there’s there’s some issues with 230 I think and like reasonable Minds can disagree on whether very narrow exceptions to 230 are warranted but 230 as a whole is good I think we have time for one more question here no it’s not gonna be a very good question I was writing so I’ll try it anyway so thank you for a very thought-provoking talk you I think make the case for a multi-faceted approach in some respects trying to address the behavior underlying it so in the case of the tribunals at Riot Games improving architecture improving user side filters you also seem to go out of your way to take deletion off the table is that what you intend to do you say deletion is not the solution of course by itself it is the solution I think there there are arguments and people are making the arguments that Alicia should be more rapid should be easier relaxing there were costs in that I don’t care for that I think there’s an oh like a too much of a focus on deletion obviously like some things gotta go you know it’s um I don’t think that there is a point in providing for instance client-side filters for child porn like that’s like they’re I don’t I don’t think we should do that deletion is in many instances the answer it is actually the answer I do think however that we we tend to focus too much on deletion and that the law especially is very much a blunt tool in this and the law focuses on deletion and imprisonment and sanctions and when we move away from the law and this is this is one of those things where you know when engineers are faced with the hard problem they put up their hands and say well that’s for the lotta to decide and when lawyers are faced with hard problems they put up their hands and say well that’s for the the tech industry to decide and yeah that I’m doing a bit of this where I’m saying like I think that it’s much more agile to work with the architecture and to look for a vast number of nuanced solutions to this problem rather than to go after it with this blunt tool you

As found on YouTube