DataFramed
DataFramed

Episode · 3 months ago

#111 The Rise of the Julia Programming Language

ABOUT THIS EPISODE

Python has dominated data science programming for the last few years, but there’s another rising star programming language seeing increased adoption and popularity—Julia.

As the fourth most popular programming language, many data teams and practitioners are turning their attention toward understanding Julia and seeing how it could benefit individual careers, business operations, and drive increased value across organizations.

Zacharias Voulgaris, PhD joins the show to talk about his experience with the Julia programming language and his perspective on the future of Julia’s widespread adoption. Zacharias is the author of Julia for Data Science. As a Data Science consultant and mentor with 10 years of international experience that includes the role of Chief Science Officer at three startups, Zacharias is an expert in data science, analytics, artificial intelligence, and information systems.

In this episode, we discuss the strengths of Julia, how data scientists can get started using Julia, how team members and leaders alike can transition to Julia, why companies are secretive about adopting Julia, the interoperability of Julia with Python and other popular programming languages, and much more.

Check out this month’s events: https://www.datacamp.com/data-driven-organizations-2022

Take the Introduction to Julia course for free!

https://www.datacamp.com/courses/introduction-to-julia

You're listening to Data Framed, a podcast by Data Camp. In this show, you'll hear all the latest trends and insights in data science. Whether you're just getting started in your data career or you're a data leader looking to scale data driven decisions in your organization. Join us for in depth discussions with data and analytics leaders at the forefront of the data revolution. Let's dive right in. Welcome to Data Framed. I'm Richie, and today we're talking about the Julia programming language. But the last few years, the big story in data science programming has been the rise of Python, which is over taken our and taking the top spot for the most popular language. However, Julia has also quietly been gaining popularity, and in the last year or two it seems to have matured certainly. I've still to hear examples of it being used by businesses to solve problems where Python wasn't a great fit for their needs. It feels like it's time to take the language seriously. So here today to tell us all about Julia is Zacharias Vulgaris. He's been the chief science officer at three startups. He's a long time Julia user and he's the author of Julia for Data Science. I'm really looking forward to hearing his views on the language. Hither Zacharias, thank you for joining yesterday on the podcast. So today with then me talking about Julia and yeah, I guess to begin with, can you just give us a little bit of context on what Julia is sure and thank you for having me here on this podcast. Well, Julia is a very particular kind of language because it is like in the middle between there's two broad categories of languages that are either high level or low level. High level languages like Python, are and mad La are very popular among people who are not very big on programming who are low level program languages like a C, C Sharp plus plus, Java and the like are very popular among developers who want to do some more efficient coding that usually lends itself to applications and anything that works on a computer basically very efficiently. So Judy tries to combine both being a fast enough like low level languages were at the same time accessible enough, which makes it more popular among people who are not very big on coding, but who know enough to code something interesting. So it is this kind of merge of the two paradigms were at the same time. It's also a marriage of two different paradigms in terms of functional languages and object oriented languages. So it is a very interesting blend of different paradigms were at the same time something of a fun and interesting project. This is really interesting. I mean, I think that Julia as being in the same category as like Python labs. All these are the sort of analyss programming languages. He's saying, it's also suitable for sort of more low level work as well, because it's got this sort of increased performance. Is that correct, That's right, that's right. Although I imagine most people, especially nowadays, they use it for high level applications like designing and implementing a machine learning model, for example, or doing some data science work that involves engineering and things like that. But if you want to make a low level application, I don't see why not. Just because it was designed with a different audience in mind, doesn't mean that it is limited only to that audience. And that's something admirable in the jutting of developers that they really expand its functionality, Like originally didn't have many libraries that made it useful for the science work, but over the years it got more and more of those because they saw there was a trend towards AI and this kind of advanced machine learning models that many people create for both in the scientific community and in the industry. So they are adapting language as time goes by to what is required. So I wouldn't be surprised if in the future they would find a niche in the systems programming as well, because Judia can't...

...do all kinds of tasks really well, really efficiently. Can you tell me a little bit more about who are the people using Julia? So you mentioned a data analysis language, but the specific roles or industries that are really using Julia, well, anyone from a researcher to a data scientist to even a that engineer I would say, could use Julia. I don't know many that engineers could use Julia, but I don't see we're not. Like one of the first applications where I got involved in Julia was doing this kind of low level data engineering work to past large amounts of text data. And you know I could have done this with c sharp because I already knew c sharp at the time. But it's Okay, let's let's try this new language. It's easier to write than She's sharp, and work is for me and other people who are in this high level kind of programming paradigm, and it was really paid off. So it appeals to this kind of people in terms of roles, but in terms of industries, I would say that appears appeals to people in data fields and also in final and also academia. Of course, so that sounds you have an interesting story in terms of how you came to be a Julia users. So he said, you start off, you moved from C sharp, so the Microsoft programming languages application development into doing things with Julia. So can you tell me a bit more about why you decided to use Julia for this? I got the thing about it. Actually I started playing around with Julia even before then. So even before I joined Microsoft as a program manager, I was very much into programming, and I had learned enough of Python to do something useful with it, and also I knew are before that. So at one point, look, I want maybe cond another languages that are seemed to be useful in this field, and I picked up Julia. Of course, I didn't go very far because we were still in Virsion zero point three or something, so there wasn't really that much to do with Julie. It was like one of these new languages that appear today and you say, oh, these are interesting, this is are very promising, but will they be around in the next five years. I didn't know that, and I had a career to worry about. So when I joined Microsoft and I saw, well, this Julia language still exists now it's zero point four, I think, and why not use it. We'll not use it in a parallel to other stuff because I wasn't obliged to use c sharp or any other particular tool in Microsoft. There people are very open minded about what tools you use, and as long as you delivered the results, you don't really they don't really care because as a program manager you'ren't expected to code that much. If you do code, that's great, that's really great. They like that. But nobody said, Okay, you have to use C sharp, or you have to use f sharp, or you have to use any other language. So that's why I said, okay, let's experiment with this. I also did a lot of Python during that time, and I said, I saw that it actually worked and over the years, even before it reached for version one point, Oh, I saw that Julia had a lot to offer in my field as well. So I said, okay, well, maybe there's much more to it than just passing large amounts of data in a data engineering fashion. Maybe there's something about predicting modeling that I can do with this language. So I explored it more, and I discovered that not only can you do lots of interesting stuff into the science, but also you can develop new models, new processes, new heuristics, what have you. And so for people who have come from like an other Python background, on our background, perhaps you can tell us a bit about how Julia is different, like how the language similar or different. There are lots of similarities in the sense that, okay, it is the code that you can look at and pretty much makes sense of it. It's a bit like Python that sense, because the Python is like pseudocode that runs Judy is not much different. If you are familiar with other languages that have this end command in their code, then it is very familiar. Like I was coming from a mud Lab background, like because I have mastered mat Lab. At one point, I was even employing see scripts in mute Lab to get more efficiency. So for me it was more intuitive, but also for someone who doesn't know about this kind of languages but they understand the structure of a high level language, it is very intuitive and easy to adopt. And we're seeing in this sex code editor...

...on this side called code Abbey, where you can tackle different programming challenges in whatever language you like. And when you I would write the Judia code, so the problem it would identify it as Python. So the syndex was close enough that that would get confused. Of course, then what doesn't know and it's not Bython, it's Julia, and it was okay fine. So they're very similar, much more similar than people may think. Now there are differences, of course, because every language has its own a uniqueness, otherwise it would pass as a flavor of a mad Lab or Python. So Julia is first of all much more efficient, and the way it handles the scripts is different, like you can't have the same function accepting different arguments, and Julia is fun with that. And they call this multiple dispatch. So instead of having a different function for tackling matrixs, for example, than the one you use for tackling specific factors. You can have the same function name and the interpret of Julia would understand Okay, well, I'm being given this input, so I think that the user must mean this, so I will call that right function instead of having different ones. That can be confusing and difficult to remember. So there are differences like that, and I don't want to go into too much about the differences, but suffice to say that it's a different language. It has more in common with functional languages, although people can use it for objectorated language programming classes and use objects and such. But if you don't want to use that and you just want to use functions, that's also fine. So it's much more similar than different. As the gist of it. It isn't in the invention mat Lab because I've noticed that the Julia syntax is incredibly close to matt Lab syntax, and that's maybe it seems like that's the strongest influence on the Julia language. So while we're talking about the properties of Julia, what do you think julius superpower is? What's the thing where you go, oh, yeah, if I want to do this task, then Julius the thing I have to use. I don't know if it's a particular task, or it's the whole environment, which allows you, even as an as a medium user who is not an expert in programming, to develop something that can run really fast. So instead of having to rely on someone else's code who may or may not maintain that code you don't know that, you can write your own code and expect it to be as fast as a library. That's not to say that you should use libraries, of course not. But if you can't find the library, are looking for some specific task, maybe a niche application or whatever, you can develop the code franscribed, build it gradually and have something powerful that you can use as a tool. If you're like trying to decide between Julia or Python, or or or made label or whatever the process. But how you would decide to just want tool another, well, it's not my decision always to make. It depends on the other people I work with. Because if everybody else in the team is comfortable with Python, I would use Python. So I would take that into account strongly, and I don't want to be like eccentric in the use of the language. If other people are petcusing Julia, great, maybe I can lure them into using it more. But if somebody doesn't like Judy, I cannot really tell them, Okay, use that because I use it. And of course it also depends on the problem domain, because if there's a very statistical problem that is tackled, maybe our is the best tool. I don't know. It is designed for this purpose anyway. It has a prethor of libraries and sophisticated the scripts around this kind of work, and the interface is pretty straightforward, so you can show it to someone who does not much programming and can understand what you're doing there with showing the notebook, for example. The same with Python. I mean, if somebody is used to using Python and they know the specific libraries, okay, maybe stick with those. I mean, if there's a particular library that's what we need, really, well, let's go for that. If we really care about efficiency and speed is of the essence, then maybe Juda's is the best option. I'm generally more agnostic now about what language to use, as long as I can get the job done quickly without creating too much free sction, I'll go that. That's...

...a very interesting point about the idea that, particularly in the corporate setting, when you're working in a team, the particular choice of language doesn't matter as much as just having a whole team agree that everyone's going to do the same things. You can sort of work in harmony. So yeah, that's very interesting. Perhaps it's different for individual developers where you get a bit more freedom in terms of choosing what you want to do. All right, So you talked about the being different libraries or packages for different tasks within Julia, So the functionality is split across into discrete areas. Can you talk us through what are the most common or most important packages for doing data science with Julia? So, first of all, data frames is essential because it allows you to import tabular data in a data frame object. There are things like decision trees to JL, psychic learn, which is popular from Python, and several other ones like emblops. I don't remember all of them by heart. But there's also a different notebook packages, but I guess we can get into that another late. And there's also the statistics package for example, which is very useful and people don't think about it. It's a very powerful library for in statistical kind of work. And there are also other packages that are more supplementary, like the p KG which stands for package, which allows you to import other packages if you're on a notebook in a notebook environment, so you don't have to go to the rebel and do the package processing there and then reload the kernel, and that can be time consuming. So these kind of packages are thinking must, but what package you need is something you can easily define on your own, like wasn't really born of that knowledge or a didn't really read one book that finds where I found all this information ready available to me? Like that. The documentation that the junior community has around the different packages is really good and really designed from a learner's perspective as well, so it's not like some impersonal kind of documentation that you think that it appeals on people who already know the language really well. Now they have designed it so that even a newcomer can understand what's happening, what languages, what packages are there, and what you can do with them, and that. Beyond that, there's also other people who are enthusiasts and judah who have websites where they expland different packages. So I wouldn't work too much about remembering which packages are there and to use them, but remember that there is a package for the stuff that we need to do in the the science and related processes that is there, even if it's not really as mature as the packages in Python, or are they are functional enough to create something useful. So I like that. The first package you mentioned was the data frames package, obviously on the Data Framed podcast with big fans of data frames, so that's really good to know. I'm curious as to what the sort of the culture is around packages. So for example, in Python the data visualization, they're just lots of different packages available. So you've got map, pot LYE and c born and plot ly and plotline and all of it and all the rest. So there's all these competing different packages. But in some cases, with for example, machine learning, you've only got really psychic learning pie Carrot. There are the two main ones. So with Julia, do you say cases where you have lots of different packages competing with each other or is it more that you just have one or two that are just the standard tool to use for a particular task. There are some packages that are competed with each other, but there're not that many. You see that more often in the visualization applications, like there there are several plotting libraries. If you don't really care so much about the devisolution, you just want to get a plot out there and perhaps exported as a PNG or whatever. I think god Fly covers you for the most part, And if you want to have the more established and more go to package that now is promoted with Julia, plots to j l is fine as well. These two packages can cover most use cases, I believe. Now if you want something more elaborate, I'm pretty sure you can find it. The people who make those packages are very happy to document them and promote them. But one thing that I really enjoyed about god Fly is that, first of all, his name is really cool, but also it was all developing Julia. Because other packages you find it okay, there are more like rupper functions around...

...pre existing packages elsewhere, but guld was developed natively entirely. And beyond that, if you're really very keen on plotly, for example, there is a plotly package as well in Julia, so you don't have to limit yourself to the packages you find in other high level languages. So in Python, package management is this sort of horrendous thing where you've got a choice of Anaconda or the pipe Pye stuff, and just keeping track of having the right packages and the right versions is always a bit of a nightmare. So what's the package management situation like in Julia. Injudia, it's much more straightforward. You just press one key when you're on the report, for example, and it takes you to the package management mode, and there are the only commance you have available to you are relatile packages. So you can first of all, list of the packages the installed, you can add a new package, you can update all the packages and things like that, and whenever you're done with it, you can just exit at more than go back to the normal problem. There's a variety of things that are really good in this particular package manager, and I believe that's one of the first things that were stackled seriously in the language. One of the creators of the language actually handled this aspect, if I remember correctly, and that's one of the strong points of version of one onwards is that they had really got this down. And by that I mean that, Okay, you don't have any problems with dependencies missing and things like that. Okay, when you install a new package, Julia recognizes that, Okay, this package needs listen that to work properly, so it installs it automatically. Also, when you when you update a package, doesn't update the package. You update everything that is that needs updating. So make sure that everything works well and together with each other. And that's that's a very strong point in my view, and the fact that you can also do that while you're on a Jupiter notebook for example, but also makes it easier so you don't have to go out of the shell and do BEP install or whatever package marriage you're using. If you want to develop your own Julia package, is that something easy to do or what you needed to get started creating your own junior package. I can speak out of experience on this part because I haven't developed a package and I don't really plan to. But I know someone who is not a developer professionally, but he knows Julia well enough that he probably knows it more than most people, and he developed a package on his own without too much concern. So from his experience I control that it is not really that difficult. Also, I've seen people develop new packages over the years, so it's not really that challenge to develop a package a new package, But I might the hardest thing would be not to make the package, but also to create enough documentation for it to make it useful, and also to maintain it because over the new versions of Julie the package may not work properly, so you need to update the code maybe at new features and functionality to it. So it is a big project. It's not something they can be taken lightly. Okay, fair enough, all right, So I'd like to talk a bit about the sort of the tool ecosystem working with Julia. So Julia is of course like the ju part of Jupiter notebooks, but Jupiter notebooks the standard way that people will write Julia code or other other tools. It depends on who you're asking, because people in my line of work prefer to work in a code notebook kind of environment or codebook as I call it, often because it's much easier. You see, there is also that way you can also demonstrate your work to someone else who who doesn't really understand programming that well, but they understand, okay, well you write this code and creates this and it's also a neat entiety. You can put the mark down as well in it, and also lots of text council put it hyperlinks and things like that, so it's like a fully blown kind of report kind of thing which also includes code so you can actually change it on the fly. So for someone in this kind of work, it makes more sense to work there. But if you want to develop something that is more coding related rather than results oriented, then perhaps an idea will be a better choice. I work with both. Jupiter Notebook, even though it's by far the most mature codebook...

...environment, is not the only option, and there are a couple of other ones that are native to Julia, like Pluto and then Neptune. And just to be clear here, we're not talking about the Neptune things that you find on a WS. The Neptun note Book is a different project in Julia which works with Julia code only interesting, so there is a rival formats. Then you're saying in a way, yes, but at the same time, I mean you can use both at the same time and not feel that you have to choose one or the other. Like there are different packages in Julia that allow you to write code in Jupiter or Neptune or whatever you like. If you want to showcase code to someone who already has Jupiter. Maybe it's best to write it in the Jupiter notebook. If you just want to write code in a notebook environment and then make a bit the eff out of it, for example, the net would work fine. And the plus of this particular option is that the nept notebook is written in in Julia code, so it's not adjacent file. It's a it's a Jupiter, it's a Neptune file, which is basically Julia file. So you can access it through an idea as well or a text editor, and you can actually use the code. Even if you don't have a Neptune a story, you don't care about Nepton, you can just take then notebook and put it on a text editor and make a script out of it, because the necessity is a script the whole thing. The drawback is that you don't see the results of what you have done. It doesn't store the results. So it sounds like the Neptune whiles are fairly close to something like our markdown or quote reporting files from the our community. So you also mentioned idea, So what are the ideas people use for writing Julia code. Whether are several ones. The ones that are quite popular depend on on on who you're asking. But this atom idea, which is popular across different languages including Bash, worlso works with Judian has a really good in a good dictionary that can recognize to your code and also alert you of potential errors. And there is also the Microsoft one two ads code is that that's right? Yes, And it used to be a different idea that was only for Julia, but now this is part of VS Visual Studio from Microsoft. So these are the main ones I think, But you can also use m if you are into that and other editors. I mean, it's not really that important what editor used, as long as you you know it well and you're comfortable with its environment, and it helps you develop code and edit code fast. Absolutely, I'm sure there's like three people listening to everyone should uce emacs and but it's a very elish choice. I think. Have you seen any success stories with people using Julia. I've seen some, yes, And the problem is that it's not really something you you read about often, Like you won't see many people on Lincoln talking about I have an amazing project in Julia unless they're usually tied to the Julia community, usually directly. But there there are lots of success stories, and this manifest usually in a particular present nations in conferences, especially the Julia corn and places like that. So there are many people who have done great work at least on this in the scientific community and elsewhere using Julia. The one small success story that I have for myself is that I have developed algorithms I wouldn't even think possible to make them in another language using Julia because they involved lots of computation, and to make them scalable and work in a deterministic, fun fun fashion is not something you can easily do elsewhere. Like I was always into making niche algorithms and diplomating them and have to try that in different languages. But after a while you realize that the speed of the language is also important. Also the fact that you can make applications easily and have them optimized using specific tools in Julia can also facilitate that. So I think the best success stories in Julia are yet to come. So you're talking about writing your own algorithms and Julia, can you give us an example of an algorithm that you wrote? Sure,...

...although I have Bobby sized them. One of them involve cybersecurity. Is a cryptographic system that handles large filts as well and make sure that they're their side there encoded in such a way that is not it's not easy at all to understand what's happening and to try to reverse engineer and break the cipher. This is what I call the Thunderstorm cipher. Beyond that, I did most of my work in data related applications. I think it was last year I made this ce script Optimal Clustering because it's a deterministic clustering algorithm that is much self directed, doesn't need much input from the user, and it can handle scenarios where there's not a very clear blob of data came in stas for example. I emplought a series of different heuristics to make that possible, and it can still use improvement, but this is something that I'm pretty content with so far. Also, another algorithm which is more more like a series of algwork just like a frameworker have developed for analyzing different kinds of variables for the engineering kind of work to facilitate this kind of processing without having to worry too much about what are the variables like, what are distributions? Are their outliers and stuff. Now, this framework harndles all that on its own, so it's gonna tart the data driven and the data different approach. I think can shine in places with tools like Julia because Julia can handle the computations really well, lots of computations really fast, so you don't have to rely too much on assumed knowledge about the data. That seems like a pretty useful feature, just like not having to make too many assumptions about what your data is like. So we talked a bit about a successive with Jupiter. I'm also curious as to where people might go wrong when they try and adopt Jupiter themselves. So if you heard about any cases where people try to use Julia but then decided it wasn't a great fit. I haven't heard any cases like that, but I can understand why some people may maybe the discouraged or deterred, but probably has to do more with the company dynamics, like if the company doesn't endors the use of new languages, for example, or it's focused on specific language because they have lots of legacy code in those I can imagine how somebody who wants to do something in Julia, even though they can there, they can't. They're not allowed to. But I don't know anyone who said, Okay, I tried Julia and it actually sucks. No, I never heard anyone like that. I have heard people who said, okay, I tried Julia, but I actually found that Lua is faster. Okay, maybe in some cases lually faster, I don't know. I mean, La is a very fast language, has been around for longer than Julia. It's always the personal application, I guess, so maybe taken on a personal level. Is there anything you wish you knew when you started learning Julia, Well, lots of things. First of all, I wish I knew that it was going to blow up so much like in the beginning, it was like an niche language. And the only book I could find about Julia, which I didn't really bother to buy, even because they only had a couple of chapters about it, where it's like introduction to some new Upper Common languages, And it had three languages at least in that book, and one of them was Julia. So you can imagine that the covering of that language was very basic, very shallow even. And yeah, I understand because somebody didn't want to put all their money in an upcoming language, which they didn't even know it was going to make it. So yeah, I wish there was there were more resources. I wish there was a bigger community when I was learning it. But at the same time, I don't know if that would have intrigued me as much in the language. I may have learned it faster, for sure, but I don't know if I would have learned it deeper. So it's really hard to think about the hypotheticals. But I definitely wish I would have meant more people in proximity to where I was who also used Julia. Like the first person I think I actually met in person who used Julia it was actually any good at It was in Seattle, at least a year or two after I started playing around with Julia. All right, So I guess now the ecosystems a little bit more mature and there are some other resources. Just if you're...

...an individual, how do you think I should get started learning about Julia? I believe I have contribulity in the resources of this episode. A particular weaklet work is a kind of collection that has a lot of material in various formats, usually links, and as you do with one link, to share lots of content. So I have developed this way as an introduction to different resources that somebody can use, most of which are for free, and and that can be a good starting point. So you self telling you are try this and the other, and you probably forget half of what I said. Just try this stuff and the wake that or most of this stuff to get not just knowledgeable about Julia, but also inspired about its potential. And another thing I would say is, okay, don't rely on just on what you learn, like theoretically intellectually, but build some muscle memory about it, because the only way to really know the language is when you start using it. Find some interesting problem that you are tackling. Even if it's already solved either by you or someone else, doesn't matter. But try to solve it in Julia and see how it goes. See where you get and how you can get unstuck. Talk to other people, even if it's in a program solving kind of fashion, to see how you can tackle different challenges in Julia. And this is how you actually can learn it better and deeper. And suppose you just want to have like your first few hours of julior coding, what are the things that you need to know first in order to get started. I think some knowledge of programming would be useful. If you don't have any of that, then startled something very simple like the simple programming challenges that you may find on racist like project Oiler and things like that, some mathematical application to get you started. Also, try to see how you can implement something you have already done elsewhere. The code that be actually is a good place to start for any language that it supports, and that's one of the places I actually I still used to to refine my programming skills. So things like that, like very trivial, trivial examples, but done properly, can really help you develop confidence and skill at the same time. Moving from think about an individual to a core processing how would you go about adopting Julia for your team? First of all, it depends on who you are what you're all is in that team. If you're the team leader, it's a bit easier because you can tell them, okay, well this is Judia. You can I can provide them with resources to learn it, and I can show them code I have development. Let them play around with it and get acquainted with it and understand it's not really something they can't handle, because I imagine in a data science team, people would already know a bit of programming already, especially if they are more senior to the scientists. So that's one approach. If I were in a team, not as a leader, but as a member of the team, I would try to showcase it with an example. Try to do something that would normally do with Python or whatever other tool I was using Intludia, and show them side by side. Look, this is the Python code, this is the Julia vote, and this is the performance in one this is the performance and the other one. And so that this can also live in a they not that we normally use. And this is a more convincing approach, I believe than telling them, okay, well look to this great look what this other person says about it. Look at this nice book about it. No, these are intellectual knowledge. But this is not convincing enough if you shouldn't that this thing actually works and I can't do it, and you don't need to go to a course, although that would be nice, but it's not something that is too far ahead, like it is something we can do readily soon. And look, there's also these libraries that you can run either in Python or in Judia to migrate code from one to the other. I should have touched upon this sooner, I guess. But there are bridge libraries between Judia, Python and R and also other places like I think there's also for c as well, so you can import Sea code in Julia and probably other ones as well. Since the time I have done this research, have even written an article about this on AI gents dot com. So if you have already code in Python, you can easily use it in Julia and vice versa, so you don't have to one day say we...

...don't use this anymore. We go to the new language and forget about the old one. This can be done gradually, so if you have a plan of this migration, then it is much more feasible and much more risk free in a way. So I really like your point about showing some sort of business impact really quickly, because that's gonna be important to make the managers sort of see that this is gonna be a viable thing. But I'd like to talk a little bit more about this interoperability. It just seem a bit like if you're adopting a new language, then it can be existing projects or there so is it. Are you saying that it's possible to have Julia and Python within the same projects the world? Like? That depends on what you mean by project, Like if it's a very large project, yes, definitely example components. Maybe certain parts of the pipeline can be done in one language and others in another one. So one language can yield its outputs in a CSV or Jason or whatever, and the other one can pick them up. Some of this functionalities can also emerge in the same notebook with the bridge package. But if you really want to go for efficiency and less maintenance overhead, it's best to have one language. So this having both languages at the same time, it's more like a temporary situation won recommended for the long term. Okay, So you're saying that ideally want to standardize like all the projects or all your work, all the analysis in your team on a single programming language. Is efficiency, Yes, and that's that's doable. I mean you can easily migrate from one language. Not easily, but it is doable to migrate from one action another, especially if they can work together. All right, And is this interoperability you mentioned with the Python is also available for the programming languages. Yes, you can have interverability between Julia and R as well as studency. And how about low level languages, I know things like Python and are. If you want to make your co go faster, you can go see your photo and or C plus, so the same truth of Julia, or is that not a necessary thing because it runs faster. It is possible to you to leverage SEE if you want, But I don't know anyone who does that, And to be honest, we're surprised there was even a bridge package between Julia and C because they're comparable and efficiency, so it doesn't make much sense to do that. It's like for someone to have a bridge package between him and SEE. Like Nime compiles in C, so it doesn't need to have a bridge package and SEE the same with Julia doesn't compile and SEE. It's completely different in terms of how it works, but it's comparable to SEE. So where's the point? Fair enough? So that seems like C is sort of not necessary to worry about for most people, all right, So I'd like to address one of those big face that a lot of people have in terms of getting started with Julia and that's that maybe the ecosystem isn't big enough. I think Julia is sort of like the fourth most popular data science language after Python. Are Lad so the company's in particularly Siderent adopting Julia. How justified of those fears that the ecosystem isn't big enough? Yet they're not really realistic fears, Like I understand where they're coming from. Okay, well you spect to to bet on a language that is more established, there's more people, more libraries and all that. But like thirty years ago, Python was like very new. Would you have thought of investing in Python back then? And if everybody thought the same way that those people with those fears think today, like nobody would have ever shifted from a fortun and see ever or Java. So the fact that there are new language is coming about over the year shows that there are people willing to take a chance on something new. And in the case of Judia, even if it's not as big an ecosystem as other more established languages, I think the risk is minimal, and partly because of the British libraries I mentioned before, and partly because it is a growing community, so it's not like other community has been staken it for the...

...past few years, or you know, you're entering some kind of niche territory that only you and a few other people are going to care about. Now, this is a growing community, and it's not like a growing community of amateurities. There's also people like vested in education who are investing in Julia. There are people who teaching court teaching courses in Julia. There are people who are doing serious projects in Judia. So it's not a hobbyist language. I think I have to make this very clear. There are languages all there, and I always like to explore new languages that come about because maybe there's the next studio out there that's now in the works. But I haven't seen anything like that, at least in the data science area. Okay, So there any sort of notable companies or organizations that are making use of Julia. I know many universities are looking into that, and they have already had courses in Julia about companies. I don't think I know many, and there's a good reason for that. Virial Sha said once I think it was in a conference, is that he his team had advised some people in the fintech industry about Julia how to optimize the use of Julia, and he found that those people didn't really want other people to know they were using Julia. That was a few years ago, so maybe now it's more well known. But if you are a company right now and you found that actually you can do things much much faster using this new technology, which is not that new. But if you have discovered that, you have proved it to your stakeholders when you go about demostrating that to everyone, even to your competitors. So I think that's how it is now with Julia. There are many companies who may be using it, but they don't really they're not very open about it yet, so that actually sort of alliance with the things of her. So the main companies have heard that we're using Julia have been involved in financial technology, but that's so secretive about what they're doing. It's very hard to find out many details on what's going on. But it seems like maybe that's where the heart of the Julia community is. So I'll not note actually like you know a little bit more about like who's involved in the Julia community. So, for example, where do you get community support if you want help with Julia well, there are different areas where you can get support, like immediate support, like you have a problem back now and you want to solve it as soon as possible. One of them is stuck overflow. It's pretty well known among all people who do any kind of coding that you know, it's a very big community of users and many of them are experts as well who offer some kind of help. Maybe it's not the answer that will they solve the problems of the way, but they would guide you was the answer, and you can have online conversations with them in text formats or other people can see that and benefit from that. So if you can't find your your question already there, you can always ask your question and get some feedback from that. Beyond that is also the discord server of Julia community, which is I haven't used it much, but I know it's there and where many people talking about anything to the related and beyond that, if you're if you're like a company especially, you can also ask the to be a computing company who provide this kind of consulting professionally, like they know the language better than anyone. So any question you have probably have heard before so you can answer it pretty well. Now beyond other thing, there's also the reddit subreddit somewhere that has Julia, so people ask questions there if you're into this platform, and I'm sure if you search beyond that you can find other places like Quara even I don't know. So last question on Julia. Are there any misconceptions about Julia that you'd like to clear up? There's this misconception that you know, just because it's fast in everything, No, that it is fast in general. Yes, it is really well done. It can handle certain computationally heavy tasks really well, but it doesn't mean that it's going to be fast in your application. Specifically, you may have a very niche kind of simulation that involves a very particular kind of data that actually it may not be the best option for you. So just like you see someone who is the really good at math, okay, that person may not be very good as physics for example, and vice versa is the same with the programming languages. Julius is good overall, and...

...there are lots of benchmarks that have done and they continually update the benchmark graphics where they show how the compare with other languages for specific simple scripts they run. But it doesn't mean that it's going to be perfect for you. So in each case, I would say approached the whole mother with a bit of a grain of salt that yes, Julia's fast, but it could be that there's another language there is faster. So if you're into programming in what experience with a different language, go for it. Don't take anyone's word for it. So before we finished, I know that you've just founded a new staff up. Perhaps you can tell us a little bit about that. Well, I'm actually more of a co founder. My co founder had the idea and started working with it, and then he got me involved and I jumped to the opportunity straight away because it was a really interesting idea and it seems to have a lot of merit, not just as technology but also commercial aspects of it. So yeah, I know one of the two co founders at the moment of this company that started in the this incubator in Amsterdam called Antler. Antler is the incubator. We don't have a name for the company yet, but we're looking into conversational AI to help training of specific professionals. So the idea is to use AI for helping people instead of replacing people, which is many other AI based companies tend to do. So it seems like a simple idea, but the complexity of it are more than meets the I for sure, but it's a very interesting challenge. It's definitely doable, and it enables us to not just get to all the technology, but also the usefulness of this technology in the real world, helping real people with real problems they face every day. So I don't know if any of the listeners are interested to work as developers or dead engineers or something along these lines with us. If so, yeah, we can always have a chat about this and develople on LinkedIn to connect and take this farther super It sounds like a very exciting new idea, and certainly I like the idea of AI being used to help people rather than to replace them. So thank you very much, Cries. I've learned a lot about Julia that has been really interesting, and good luck with your new startup. Thank you very much and thanks for having me. You've been listening to Data Framed, a podcast by data Camp. Keep connected with us by subscribing to the show in your favorite podcast player. Please give us a rating, leave a comment, and share episodes you love that helps us keep delivering insights into all things data. Thanks for listening, Until next time.

In-Stream Audio Search

NEW

Search across all episodes within this podcast

Episodes (121)