I just read about dask earlier today, very neat project! riko already handles parallel processing [1] but adding distributed processing sounds tempting. TBH though, distribution isn't high on the priority list. But I'll be happy to accept a PR if you are so inclined :)
I think there's a general misconception that Julia is a DS/scientific programming focused language. It just so happens that these are areas where its advantages are most apparent, and give the biggest early wins. Given enough time, though, I think the paradigms that Julia is exploring will make it a killer general-purpose language.
As for Julia 2.0, it was decided to put off implementation of traits and interfaces until then. That said, some of the ideas that were being bounced around during the post-con Hack day in these areas are really exciting. In short: Julia may be the first language to pull off behavioral typing in a practically usable way.
"Given enough time, though, I think the paradigms that Julia is exploring will make it a killer general-purpose language."
I think its already a Killer general-purpose language (except for the module system).
I'm just not sure if it is good enough to unseat incumbents when there are things like rust with its deterministic memory management or python with all its momentum and compiler technology coming along.
The thing about Julia is that it's playing a whole different ballgame than pretty much every other popular language today...it's just that most people don't realize it yet! The work and design that's gone into Julia's type system and multiple dispatch has the potential to be a real game-changer, but outside of a core community putting it to good use, I don't think it's potential has been fully realized.
This is also why I'm excited by what's coming next in v2.0. Some of the early ideas being explored at JuliaCon with respect to traits and interfaces will begin to really shine a spotlight on its true power.
> The thing about Julia is that it's playing a whole different ballgame than pretty much every other popular language today
I've noticed this. I've been using it the past four years for much of my dissertation work (starting right when it was released). Julia's type system has the potential for some extremely cool things.
I've been toying around with the idea of making a package that focuses on runtime static typing. This would be particularly useful when using the language interactively (like in a Jupyter notebook). The idea is to perform a check at runtime to make sure all variables in a function are assigned an immutable, concrete (non-abstract) type, and then compile an optimized version of that function on-the-fly . One source of pain in a lot of my Julia code is that unintentional type instability contributes to a lot of unnecessary performance penalties, and it takes quite a bit of poking and prodding before I figure out exactly which line is responsible. Forcing a check over the function would prevent these occurrences.
I also seem to have the problem of inadvertently calling functions that allocate and deallocate tiny amounts of memory on the innermost for loops.
(Then again, maybe nobody else has these issues and I'm just bad at deducing when AbstractVector can't be used.)
It is unmaintained now, but you might want to have a look at [1], which builds on the type inference system to add some static checks. I believe there may also be some related work in [2].
> Then again, maybe nobody else has these issues and I'm just bad at deducing when AbstractVector can't be used.
If you're familiar with duck typing, behavioral typing is (in essence) the reification of duck typing in a concrete type system. In other words, instead of specifying the type of your argument as "Array", you could specify "some type that is indexable, iterable, and can be appended to".
In Julia (mind you this was just the idea I saw being considered), today you would do:
function foo(myarray::AbstractArray)
...
end
but in the future you might be able to do something like:
function foo(myarray::ANY{getindex(), setindex(), iterate(), append()})
...
end
what's really neat, though, is combining this with type aliases, you could have:
typealias Arraylike ANY{getindex(), setindex(), iterate(), append()}
function foo(myarray::Arraylike)
...
end
The difference between structural typing and behavioral typing is subtle. In Scala, you're essentially asking the type system to check for the presence of a member function on the object with a specific type signature. Since the function is contained within the object, you're still only typing based on the object's structure (hence: structural typing).
In Julia, objects are data-only and methods are defined at a module level. So, whereas Scala's structural typing need only introspect the object being passed as an argument, Julia's behavioral typing requires introspection of the entire dispatch tree. The benefit to behavior typing and Julia's multi-dispatch is that if you are missing one or two methods for some type in order to be able to use it in some function, you can always define the missing methods locally.
You could also view C++ templates as a kind of behavioral typing, although the experience of using them in that way is not very pleasant. Concepts should help clean that up when they finally arrive in the spec.
It seems like the weak spot for using Julia outside of scientific computing is deployment to interesting targets.
For example, with Go you can build a standalone executable in many platforms, and other languages run on interesting targets like browsers or phones, or run on the JVM.
Yes. Module pre-compilation is already possible, and last I heard there is work underway to use the same sort of mechanism to allow for single-executable compilation. The problem, of course, is that Julia and Go are very different beasts, and pre-compilation will almost certainly limit some of the currently available dynamism. (Already there are packages that cannot be pre-compiled due to these limitations.)
Of course, in this regard Julia is not any different than Python, Ruby, JS, PHP, etc. Also, it's worth noting that Julia has cluster-computing as a concept baked into the language. So, whereas you might need to worry about distributing Go executables to multiple machines in a cluster, with Julia you need only have the Julia runtime installed, and from there any Julia program can distribute itself to any available nodes without further action required.
Does this obviate the issue that one has to assign the definition of meaning somewhere in the graph? ie Meaning is subjective, and it has to be inserted to propagate down.
Remember that the other thing about Python is that it's got good support for other types of programming -- for example, I work at a company which settled on Python (in part) because it allows a single language end-to-end. Our data ingestion, analytics/processing/number crunching, and end-user applications based on the data can all be built in Python, and even if you're not a domain expert in one of those areas you can at least read code to see what's going on and not be totally lost.
It's like you're recommending a restaurant by saying that its food is "edible".
All in all, this is a bad recommendation. Julia is version 0.4. Its syntax changes with every version. It is not going to be stable soon. It is so far from stable that it doesn't have a plan for what stability will look like.
It's fine if you want to get in on the ground floor of a programming language, or to learn a new language for the fun of it, but it's completely unreasonable to suggest that you could replace a programming language that people use for their jobs that way.
I've seen people saying this for years, yet I don't know anyone that has actually switched over. The people I do who have tried came right back to Python, often within a few days or weeks.
I can't overplay the importance of a octave like DSL.
Julia is gaining marketshare and mindshare among grad student not just due to its speed, but because it is a more fun and intuitive environment in which to code mathy stuff.
These people will in turn filter into industry and if not them, then atleast their code.
Also macros. As Julia gains more utility for run of the mill data science, Its Dplyr like DSL abilities will be very attractive.
Do you see this type system and generic function library as useful for general purpose programming as well? How would that play with mypy and type hints?
