Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
Okay, story time.
Back during 2016, I decided to do a little experiment to test the viability of multithreading in a JavaScript server stack, and I'm not talking about the Node.js way of queuing I/O on background threads, or about WebWorkers that box and convert your arguments to JSON and back during a simple call across two JS contexts.
I'm talking about JavaScript code running concurrently on all cores. I'm talking about replacing the god-awful single-threaded event loop of ECMAScript – the biggest bottleneck in software history – with an honest-to-god, lock-free thread-pool scheduler that executes JS code in parallel, on all cores.
I'm talking about concurrent access to shared mutable state – a big, rightfully-hated mess when done badly – in JavaScript.
This rant is about the many mistakes I made at the time, specifically the biggest – but not the first – of which: publishing some preliminary results very early on.
Every time I showed my work to a JavaScript developer, I'd get negative feedback. Like, unjustified hatred and immediate denial, or outright rejection of the entire concept. Some were even adamantly trying to discourage me from this project.
So I posted a sarcastic question to the Software Engineering Stack Exchange, which was originally worded differently to reflect my frustration, but was later edited by mods to be more serious.
You can see the responses for yourself here: https://goo.gl/poHKpK
Most of the serious answers were along the lines of "multithreading is hard". The top voted response started with this statement: "1) Multithreading is extremely hard, and unfortunately the way you've presented this idea so far implies you're severely underestimating how hard it is."
While I'll admit that my presentation was initially lacking, I later made an entire page to explain the synchronisation mechanism in place, and you can read more about it here, if you're interested:
http://nexusjs.com/architecture/
But what really shocked me was that I had never understood the mindset that all the naysayers adopted until I read that response.
Because the bottom-line of that entire response is an argument: an argument against change.
The average JavaScript developer doesn't want a multithreaded server platform for JavaScript because it means a change of the status quo.
And this is exactly why I started this project. I wanted a highly performant JavaScript platform for servers that's more suitable for real-time applications like transcoding, video streaming, and machine learning.
Nexus does not and will not hold your hand. It will not repeat Node's mistakes and give you nice ways to shoot yourself in the foot later, like `process.on('uncaughtException', ...)` for a catch-all global error handling solution.
No, an uncaught exception will be dealt with like any other self-respecting language: by not ignoring the problem and pretending it doesn't exist. If you write bad code, your program will crash, and you can't rectify a bug in your code by ignoring its presence entirely and using duct tape to scrape something together.
Back on the topic of multithreading, though. Multithreading is known to be hard, that's true. But how do you deal with a difficult solution? You simplify it and break it down, not just disregard it completely; because multithreading has its great advantages, too.
Like, how about we talk performance?
How about distributed algorithms that don't waste 40% of their computing power on agent communication and pointless overhead (like the serialisation/deserialisation of messages across the execution boundary for every single call)?
How about vertical scaling without forking the entire address space (and thus multiplying your application's memory consumption by the number of cores you wish to use)?
How about utilising logical CPUs to the fullest extent, and allowing them to execute JavaScript? Something that isn't even possible with the current model implemented by Node?
Some will say that the performance gains aren't worth the risk. That the possibility of race conditions and deadlocks aren't worth it.
That's the point of cooperative multithreading. It is a way to smartly work around these issues.
If you use promises, they will execute in parallel, to the best of the scheduler's abilities, and if you chain them then they will run consecutively as planned according to their dependency graph.
If your code doesn't access global variables or shared closure variables, or your promises only deal with their provided inputs without side-effects, then no contention will *ever* occur.
If you only read and never modify globals, no contention will ever occur.
Are you seeing the same trend I'm seeing?
Good JavaScript programming practices miraculously coincide with the best practices of thread-safety.
When someone says we shouldn't use multithreading because it's hard, do you know what I like to say to that?
"To multithread, you need a pair."18 -
Biggest challenge I overcame as dev? One of many.
Avoiding a life sentence when the 'powers that be' targeted one of my libraries for the root cause of system performance issues and I didn't correct that accusation with a flame thrower.
What the accusation? What I named the library. Yep. The *name* was causing every single problem in the system.
Panorama (very, very expensive APM system at the time) identified my library in it's analysis, the calls to/from SQLServer was the bottleneck
We had one of Panorama's engineers on-site and he asked what (not the actual name) MyLibrary was and (I'll preface I did not know or involved in any of the so-called 'research') a crack team of developers+managers researched the system thoroughly and found MyLibrary was used in just about every project. I wrote the .Net 1.1 MyLibrary as a mini-ORM to simplify the execution of database code (stored procs, etc) and gracefully handle+log database exceptions (auto-logged details such as the target db, stored procedure name, parameter values, etc, everything you'd need to troubleshoot database errors). This was before Dapper and the other fancy tools used by kids these days.
By the time the news got to me, there was a team cobbled together who's only focus was to remove any/every trace of MyLibrary from the code base. Using Waterfall, they calculated it would take at least a year to remove+replace MyLibrary with the equivalent ADO.Net plumbing.
In a department wide meeting:
DeptMgr: "This day forward, no one is to use MyLibrary to access the database! It's slow, unprofessionally named, and the root cause of all the database issues."
Me: "What about MyLibrary is slow? It's excecuting standard the ADO.Net code. Only extra bit of code is the exception handling to capture the details when the exception is logged."
DeptMgr: "We've spent the last 6 weeks with the Panorama engineer and he's identified MyLibrary as the cause. Company has spent over $100,000 on this software and we have to make fact based decisions. Look at this slide ... "
<DeptMgr shows a histogram of the stacktrace, showing MyLibrary as the slowest>
Me: "You do realize that the execution time is the database call itself, not the code. In that example, the invoice call, it's the stored procedure that taking 5 seconds, not MyLibrary."
<at this point, DeptMgr is getting red-face mad>
AreaMgr: "Yes...yes...but if we stopped using MyLibrary, removing the unnecessary layers, will make the code run faster."
<typical headknodd-ers knod their heads in agreement>
Dev01: "The loading of MyLibrary takes CPU cycles away from code that supports our customers. Every CPU cycle counts."
<headknod-ding continues>
Me: "I'm really confused. Maybe I'm looking at the data wrong. On the slide where you highlighted all the bottlenecks, the histogram shows the latency is the database, I mean...it's right there, in red. Am I looking at it wrong?"
<this was meeting with 20+ other devs, mgrs, a VP, the Panorama engineer>
DeptMgr: "Yes you are! I know MyLibrary is your baby. You need to check your ego at the door and face the facts. Your MyLibrary is a failed experiment and needs to be exterminated from this system!"
Fast forward 9 months, maybe 50% of the projects updated, come across the documentation left from the Panorama. Even after the removal of MyLibrary, there was zero increases in performance. The engineer recommended DBAs start optimizing their indexes and other N+1 problems discovered. I decide to ask the developer who lead the re-write.
Me: "I see that removing MyLibrary did nothing to improve performance."
Dev: "Yes, DeptMgr was pissed. He was ready to throw the Panorama engineer out a window when he said the problems were in the database all along. Didn't you say that?"
Me: "Um, so is this re-write project dead?"
Dev: "No. Removing MyLibrary introduced all kinds of bugs. All the boilerplate ADO.Net code caused a lot of unhandled exceptions, then we had to go back and write exception handling code."
Me: "What a failure. What dipshit would think writing more code leads to less bugs?"
Dev: "I know, I know. We're so far behind schedule. We had to come up with something. I ended up writing a library to make replacing MyLibrary easier. I called it KnightRider. Like the TV show. Everyone is excited to speed up their code with KnightRider. Same method names, same exception handling. All we have to do is replace MyLibrary with KnightRider and we're done."
Me: "Won't the bottlenecks then point to KnightRider?"
Dev: "Meh, not my problem. Panorama meets primarily with the DBAs and the networking team now. I doubt we ever use Panorama to look at our C# code."
Needless to say, I was (still) pissed that they had used MyLibrary as dirty word and a scapegoat for months when they *knew* where the problems were. Pissed enough for a flamethrower? Maybe.6 -
First day on my first job ever, the boss asks me what I want to do. I indicated that I had some experience with php and the yii framework (which was at some point very cool xD), so I wanted to start with something like that. And so it goes: after two days of watching laracasts (which is an awesome platform by the way! :O) I got assigned to a project.
Now the company I work at uses some kind of self built system that tracks how many hours are spent on which project, and compares that to how many hours was estimated implementing a feature would take. That's cool, but then I saw that for the project I was working on the time estimated was 5 work days. This was the estimate for both designing the interfaces and implementing both front and backend. I knew in advance that this was probably way to little time for me, but didn't want to come over as the new kid who can't do shit x)
Anyway, I started on the project and was having fun, but the biggest time consuming aspect of the project was not necessarily that I didn't have enough experience: it was that the developer who started this project and made most of the design choices had written some very messy code, without tests or apparently any refactoring. Also, everything was extremly inconsistent and not according to all the best practices I just watched in my laracasts spree.
So fastforward a little: we're way over the estimated hours. Yay. Now suddenly the boss comes by with an almost angry face that the client is becoming angry and we need to finish soon. He makes it entirely our (me and the front end guy) problem and I just decide to say nothing and try to work faster.
Now I'm stuck writing fugly code on top of more fugly code and when I mentioned to my front end guy that I was almost finished with feature but I only needed to finish up the tests, he said something like "oh just don't write tests, that'll take too long"... Is that really the mindset of this company?! No wonder the project I work on was in a very bad state.
Thanks to devrant I see now that I just need to say something if I know that I won't be able to complete something in a certain amount of time and that other people are just like me (thank god). :) I think I'll need to post more rants to vent my frustrations x)5 -
I feel like if I ever get a professional programming job my biggest problem will be I can't work on projects with other people when it comes to programming I really don't know how to split up the work any tips?5
-
The scary thing about burnout is that you usually don't realize you are burning out before it's too late.
Personally, at least, I've worked on projects that just felt a little intense at the time, but after taking a step back due to holidays or hitting some milestone I realize I never want to have anything to do with it ever again. One project makes my stomach drop even today every time I see the code; Not because the code is bad, but how it takes me back to how miserable I was without admitting it to myself.
The biggest red flag I look for is when I'm tempted to work on stuff in my free time. When this starts seeming like a solution there's a serious problem with the project that needs to be addressed.2 -
This is a story about my disappointment in modern GUI editors for desktop applications.
Well, first of all, I grew up with Delphi 5. Delphi has an awesome form editor. It's intuitive and works without any problem. It always does what you want it to do. Prototyping is really a problem of seconds here, even for people that never used it (I guess).
But the problem is that it is Delphi. Its so old, bloated, and most problems you'll ever have have been solved (through a hack) 20 years ago in some weird forum.
So I looked on and tried many other drag'n'drop gui editors.
The one for java is the biggest pile of crap I've ever seen. It slows down eclipse /intellij and does almost never do what I want. At least its not really intuitive.
Right after that, the one for C# (this xml Designer ) is okay-ish, but it's also not really intuitive and does not always what the user wants.
I also tried other ones. But I still miss an intuitive one that works without weird side effects.
I now can understand why the Web dev stack grows in the region of desktop apps. I can prototype stuff even faster in angular than in Delphi.
But shouldn't we improve the desktop stack instead of taking some bloated stack using a language that should have never existed?9 -
Oh my god, GDScript is the single biggest piece of shit scripting language I have ever witnessed. It somehow manages to combine the very worst things of dynamic typing with the downsides of static typing, all in one bundle of utter shit
Imagine you have two game object scripts that want to reference each other, e.g. by calling each others methods.
Well you're outta fucking luck because scripts CANNOT have cyclic references. Not even fucking *type hints* can be cyclic between scripts. Okay no problem, since GDScript is loosely based of Python I can surely just call my method out of the blue without type hints and have it look it up by name. Nope! Not even with the inefficient as fuck `call` method that does a completely dynamic-at-runtime fuck-compile-time-we-script-in-this-bitch function call can find the function. Why? Because the variable that holds a reference to my other script is assumed to be of type Node. The very base class of everything
So not only is the optional typing colossal garbage. You cant even do a fucking dynamic function call because this piece of shit is just C++ in Pyhtons clothing. And nothing against C++ (first time I said that). At least c++ lets me call a fucking function8 -
You're working on a project, your boss has set you tasks to complete. You complete all the tasks and make pull requests of all the work you've done. Your boss is busy and you don't know what to do next? What do you do? Is it unethical to work on personal projects in work hours, because you're not being paid for that? How do you find other stuff to do?10
-
Disclaimer: This is all theoretical. Neither me nor my friend (with whom I discussed this) are stupid enough to even try to pursue this, but as an idea, i believe it might generate cool/new ideas/ways for handling secure communications across social groups.
Let's do some role play. Let's design a delivery app for drug dealers, think Seamless or Uber Eats, but for drugs. Not for big deliveries, like kilograms of coke, but smaller stuff. Maybe a few grams of it or something. The clients could rate dealers, and vide-versa. This would build a level of trust within the system. There would be no names, just anonymous reviews, ratings, and prices. Only the info you'd need to know.
The biggest (only?) problem we found (besides legality) was that, how would you prove that you're a client and not a snitch (or cop). This would have to somehow be handled both on signup, as well as when ordering (let's imagine that all who are clients are pure and won't ever snitch).
One of the ways we found to combat this was to have the app invite-only. This would, in theory, do away with the problem of having snitches signing up. However, what if the phone got stolen/breached by a snitch, and they also got full access to the account. One way we thought we could combat this would be with a "dispose number" or something similar. Basically, you call a number, or send a text, or message a Signal bot etc, which would lead to the account's instant termination, no traces of that user left. Hence, a dispose number.
The flow of the app would be as follows:
A client wants some amount of heroin. He opens the app, searches for a dealer, sends the him the desired amount, and in return gets back a price from the dealer. If both parties agree on the amount and price, the deal would start.
The app would then select a random time (taken from the client's selected timeframe and the dealer's "open" time) and a location (within a certain radius of both them, somewhere in between them both for convenience). If both of them accept the time and place, they'll have to meet up at said time and place.
The actual delivery could also be done using two dead drops - the client drops the money at one of them, the dealer drops the goods at the other one. Yes, this might be subject to abuse, but it wouldn't be that bad. I doubt that clients would make huge orders to unknown/badly rated dealers, as well as dealers accepting offers from badly rated clients. My idea is that they would start small, just so if they do lose their money/goods, the actual loss wouldn't be as big for them, but for the other party, having bad ratings would mean less clients willing to buy or dealers willing to sell.
A third way would be to use crypto, but the reason I left this as the last one is because it's not that wide-spread yet, at least not in local drug dealing. With this method, the client would initiate the order, the crypto would be sent to either the dealer or an escrow account, the dealer would then drop the goods at a random place and let the client know where to go to get them. After the client has gotten the goods, they could both review/rate the quality as well as the overall experience with that dealer, which would either make or break the dealer's upcoming deals. This would be pretty much like other DNM's, but on a local scale, making deliveries faster.
So far, this would seem like something that would work. Are there any ideas that might improve this? Anything that might make things more secure/anonymous?
My reason for this post is to spark a conversation about security and anonymity, not to endorse drugs or other illegal stuff.
Cheers!
PS. Really loving the new PC design of devRant14 -
This is more of an essay than a rant. TLDR at the end. I simply can't choose from all the shitty lecturers I've had, so I'm going to have to go through them one by one. But of background. I'm currently in 7th year of college, I did a multimedia degree in 2 years, a intro course to Software Dev and I'm currently in my final year of my Software Dev degree. So let's start.
Intro Software Course
- we had a database module, which was thought by, I shit you not, the head of the psychology course in the college, she attempted to teach us Databases using access. And not even using SQL, using access GUI components and it's query builder. Need I say more?
1st year software dev
- We had a networking module, the guy that taught the labs, he literally didn't say more than 12 words the entire 12 week semester, his answer to any question you asked him was a grunt and "research it"
- We had a psychology module, I have no fucking idea why, but instead of learning something useful we were told to read this and get in touch with your feelings...
- database module. Yes we actually did SQL here, 12 weeks of select statements and normal form, talked about by a guy in a monotone voice, who sounded like he was contemplating bringing in an assault riffle some day. Also instead of using MySQL he decided to use Ingres. Why I will never know.
2nd Year Software Dev
- We had a module called Algorithms and Data Structures. The lecturer gave us problems she couldn't solve. Simple problems. She was also crazy. Absolutely nuts.
- Object Orientated Programming. I had this lecturer for 3 semesters up until 3rd year. This guy did COBOLT in college, graduated in the 70s or something and went straight into teaching, he taught us Java for nearly 2 years. He literally copied and pasted texts from PDFs and read through them in class. He told myself and another guy at one stage he really didn't care, and was just counting down the days to his retirement.
- Databases again, different lecturer from 1st year, taught us for 2 semesters (24 weeks) and somehow managed to teach us nothing.
3rd Year Software Dev
- software engineering.. This is where the biggest cunt I've ever met was introduced. He arrives into class 15 minutes late every time without fail, talks shit about stuff that has no relevancy to the topic at all, tries to turn everything into a rugby metaphor and every time you ask a question he somehow dodges it and swiftly changes topic. This cunts past profession? A Project Manager. Fucking typical. This dickhead has also thought me 2 other modules.
4th yr Software Dev
- El cunto mentioned above for 2 more modules. Need I say more.
- real time systems, this module took the piss, the module was written by the lecturer which is what earns his space here. Assignments given to us, which required more time to do than we had in labs so we had to work at home, the problem we that is we were using an obscure RTOS called OS9 which would only work on the college computers. When brought to the lecturers attention he just said "figure it out"
Internet of Things - There was 2 lecturers, each lecturer seemingly working off a different plan, one week you'd have one lecturer, the next would be the other one going on about something completely different and unrelated to anything else we'd done.
Some lecturers didn't even make this list as I couldn't be bothered trying to think back about how shit other ones were. These were the ones that always stood out in my mind.
My main take away point from this is that you go to college for the paper which says you have a degree. Learning things that are going to benefit you in a career is up to yourself.
TLDR; 90% of my college lectures were shit. You need to learn useful stuff yourself.1
Top Tags
Weekly Rant
View