In the context of their SNSF-funded project  “Convergence versus Divergence? Text-as-Data and Network Analysis of International Economic Law Treaties and Tribunals”, Prof. Joost Pauwelyn and colleagues have set up a database of economic law cases that currently comprises 15,000 judicial decisions and further texts in a form suitable for machine processing. Still considered new, challenging, but also controversial in (European) legal scholarship, empirical research carries nonetheless great potential since it gives insights into what is actually happening on the ground. The article “The Data-Driven Future of International Economic Law” explores the added value of using data-driven research as compared to other more traditional methods, but also the challenges of engaging in this type of research. More details with Prof. Pauwelyn.

You argued in your article that those who engage in data-driven research need to develop specific skills. Could you explain the reasons behind this statement, as well as your personal experience with this type of research?

For me it is still a real challenge to find the right people to work with. For instance, Wolfgang Aschner was a master (and then PhD) student here with a Political Science background. I did my sabbatical in Stanford where I got into these methods. I am not very technical, so we sent Wolfgang there for a year to do a master and to gain all the skills, to play with the data and to put it into the right format. I was able to rely on him for processing the data. Now he took up an assistant professorship in Canada, which is great for him, but left us in need for someone else with technical skills to take over, Weiwei Zhang, also a post-doc at the Graduate Institute. It is impossible to be an expert in all of these fields. For me, it has been a real challenge to find the people with the technical skills to scrape the data, put it in a workable form in a database that updates itself and then find the people – often different people – who know how to apply network analysis or regression analysis to the data that has been collected. At the same time these experts, who have the technical skills, do not always have the substantive insights into the legal regimes, or they do not know what legal questions or what substantive questions to ask. So, it only works if you combine the technicians who can help you collect the data in some efficient manner with those who can ask the “right questions”.

Even for our project we hired a computer scientist who could get the data in a much faster way than just typing it over. “Technicians” have the skills to do that in a very limited time frame but you need to have the money to do that, and we used the SNSF money to do that. And then you might need another set of people like James Hollway – he is an expert in network analysis, and we worked with him on some papers (see “The Trade Regime as a Complex Adaptive System”).

It is really intensive to do this type of research. It is much easier to just sit down and read a case, a treaty, and write an article about it. You also need to work with co-authors. Yet the current way academia is set up, it’s not made easy to work together on projects or when you co-publish, you do not get the standard credits for it.

How would describe the role of the person who has the substantive knowledge of the legal system in relation to the person processing the data? Is there a degree of importance?

Both roles are crucial. What I have learned is that you need to think hard about the data you want to collect, how you want to collect it, and how to make sure that the data keeps itself updated. The computer scientist in our project had to write computer codes, algorithms to automatically extract the data from the WTO website and a number of other websites. Some people have done it manually, and they literally spent years collecting data and when they had no more money, they stopped, and the data they had collected became outdated, whereas with the methods one has now, it is much faster, much easier to collect data, keep it automatically updated and then to do something with it.

Having the data is one thing, but it is not all. Now, we are spending a lot of time making the interface user-friendly so that, when we make the data public, the public, including other researchers or practitioners, say, from developing countries, will be able to use it, and do something useful with it.

How will these new working methods affect the evolution of legal research, not only on trade law, but more generally on international law? 

For sure, the human factor is still crucial. As I said, you need to look at the data and ask the right questions. But, especially in Europe, most legal scholarship is produced by “sitting down, reading a number of treaties or cases, highly selective, and then writing normative views about how the world should be”, which is fine, but it would also be useful to look at what is actually happening on the ground and look at all treaties, all cases, in order to find some broader patterns. And now, we turn again to the initial question, the human factor is still necessary for distinguishing between noise and the real patterns, and explaining those patterns and why one needs to reform a regime or not, and how, learning from the past.

For instance, on a recent WTO Public Forum panel we presented some work we have done on WTO dispute settlement, more particularly with regard to the fact that they “feel busier than ever”. However, the number of cases filed is actually going down, yet the number of cases going to the panel stage is relatively steady. This has to do with settlement, and lower settlement rates. Lower settlement rates in turn have to do with the increasing number of trade remedy cases, and trade remedy cases cannot be settled under most domestic laws, so that partially explains reducing settlement rates. The point is, you need to find the data and solve the puzzle the data raises, and one does this by human thinking and substantive knowledge. It is a constant back and forth between the data and the substantive knowledge, between using the data and trying to explain or making sense of it.

Resourceful players like the European Union and the United States may have lawyers who could look at all the treaties and cases and come to the same insights. But if you come from a developing country the availability of that data will allow even for departments of one or two lawyers, by using their computers and that database, to do a lot more, helping them better negotiate their treaties or litigate their next case. If anything, the availability of this data democratises the insights one has when analysing this data. The other challenge is to make sure that when researchers collect data, this data is publicly shared.

How did you come to develop this interest in empirical research? Is it because of your particular area of expertise? Or were there other triggers?

To be honest, if you are a legal scholar and teach in the United States (as I did in the beginning of my academic career at Duke Law, from 2002 to 2007), you teach in a very empirically minded environment. You are almost pushed to do something in that sense whereas in Europe the tradition is different. I got particularly interested in empirical research when more data became available in the fields I was working on (trade, investment) and machines could collect the data and give insights into them. Until now, it was political scientists who would have the project money as well as teaching assistants to do this. Law professors actually do not traditionally do this. But now, it is much easier to collect data and to “play around with it” than it used to be. As I said, spending time in the United States and in the practice of international law, one sees that law as a discipline and its normative focus are really put into question so that you are pushed to look at the real world and use data. So, that is probably where my interest came from.

SNSF press release > 
Blog post by the authors on EJIL: Talk! >

Full citation of the article: Alschner, Wolfgang, Joost Pauwelyn and Sergio Puig. “The Data-Driven Future of International Economic Law.” Journal of International Economic Law 20, no. 2 (June 2017): 217–231. doi.org/10.1093/jiel/jgx020.

Interview conducted by Oana Ichim, PhD Student in International Law