I’ve been very fed up with AI for about past 6 months, completely started to boycott it, discuss it with people of all kinds of different views, and I found joy and pride once again in my own work.
But the world issues coming from AI, and even more so from the billionaires and empires behind it seem to further pile up to insane heights. I’ve been trying to learn more and more about it, and after my bachelor and masters I am considering pursuing phd and research surrounding AI, especially from the critical perspective, which seems to be deeply neglected in the research pov.
This is still a few years in the future, and lot could change, but I am curious what do people here think about pursuing such a thing, and if in current academic world it is even something that would be possible doing, given lot of the grants and funding of AI research comes from these companies that just want to gain even more power through it.
As I said, I am already trying to know as much as possible about it, but I would like to look more deeply into its impact on society, impact on students etc.
Do you think this would be a worthwile endevaour? And if not, where do you think I should be heading to make change about this while not completely starving to death?
I am currently studying Cybersecurity as my bachelor, so my background is very technical… I think I would like to go the technical route, but also sprinkle in a bit of sociology, or maybe philosophy or something politics related… Maybe focusing on being sort of bridge in interdisciplinary research with deep knowledge of the technical aspects, while also focusing on more social aspects and collaborating on research with people that have deeper understanding of these issues. There is a master degree in Prague called prg.ai I believe, which kinda tries to teach about AI from many different povs, which could be interesting for me.
To understand things from a technical point of view, you’ll want a solid understanding of Artificial Neural Networks (ANN) and how they are usually trained (back-propagation). These have been around for decades and are also used for classification tasks and things like image processing in your phone or GPU.
To create an LLM, you feed the output of the ANN back into the network, creating a Recurrent Neural Network (RNN) and you tokenize the input.
It also helps to have some general knowledge of machine learning (ML) and understand terms like supervised learning and unsupervised learning.
Most of the ideas behind LLMs have been around for years or decades. The big difference is that nowadays we can throw enough compute at the problem to make them viable.
There are two ways to make ANNs and LLMs more efficient: better algorithms and more specialized hardware. For example, bitnets with ternary connections run much better on conventional CPUs and GPUs. Hardware like Google’s NPUs, OpenAI’s new chip, analog computers or neuromorphic computing could run them more efficiently.