Nectarine

Hey there, I see that Ishaan is intereseted in helping moving demovibes source code forward

Is there a possibility to put something on a test/preprod server in order to be able to test new source code with real data ? Maybe with the help of communicator ?

What do you think about this nyingen ?

I don't want to move too fast because data migration can be headhach. For now I never modified database structure from the original demovibes repo.

I think that should be possible; there is a dev server already that has a (slightly out of date?) copy of the database. Getting a necta beta set up on that server shouldn't be a problem, I imagine.

Depending on the db structure, it might not be necessary to modify it. It might be good enough.
However, if the structure is bad, the first step would be to redesign the business logic, the "flow" of the app, and to map models on this flow. Then we'll structure the data to serve the business flow (and optimize it, if needed, to improve resources management and efficiency on the db). This mean we could temporarily rely on orm to save dev time. An orm that work with db mapper pattern instead of active record so we have more flexibility with data structure.

It'll always be time to migrate data (wherever it is required) using a command line script or import mechanism usong data mappers.

And big files are a problem in the sense that you'll have a lot of coupling AND a lot of painful refactoring coming the moment you want to split properly.

The idea here is to bring constructive feedback, not to criticize for the sake of it.
Ideally, the code would be in a language and using framework that are widely used and mastered so mote people could add pull requests and contribute.
Having an open ticket system were people could take a ticket and propose code would be great.

Regarding infrastructure, I could ask a devops in my team to work on this too to propose something, if needed.
Asking one or two people to handle the whole thing is too painful.
We'd need to lake it an open sourced project, with a set of qualified reviewers (aka professional of the field and the server owners playing the role of product owners)

That's great

Code is open and available here : https://framagit.org/demovibes/demovibes
I already created some issues, for migration or enhancement.

For data structure (database), I didn't touch anything not because it was good or bad but because migration will be easier. The main problem was that demovibes source code was using an obsolete version of Django and python2).

Ideally, being able to have a server where code could be automatically deployed would be sufficient I think.

Actual demovibes production source code has been modified by nyingen (and maybe others ?) to add some features and correct some bugs. We'll need to get this part of code to be able to push everything on production.

So :
1. I upgraded lot of stuff and simplified some
2. I think it's time to put it somewhere to get help (being alone can be painfull)
3. With the help of nyingen (and others), we'll be able to upgrade and protect this part of internet and demoscene

EDIT: And clearly, actual source code is not sufficient because I worked only on the website part, not the stream part. And without sufficient datas I'm not able to test it correctly for now.

Mindiell, my only concern now is the choice of Python and Django to be honest.
We should have moved away from both for a revamp as they have a small developer base and are not the optimal choice (does not matter how much one likes python, it is better suited for AI now than for pure web dev).
I am worried that putting effort in another django version is only shooting ourselves in the foot as it'll be another version that will never be updated again as nobody will be willing to put in the effort.
Had we had nodejs (any framework of your choice), or PHP (Symfony for example), find contributors would be piece of cake. Find tools for dev would be piece of cake... Even Go would have felt more suitable for the backend coupled with any front framework...

While I would not hesitate to use python for AI, or blockchain, or data pipelines, I feel that it is not the best suited for web applications...
And so far, practical experience has concluded the same (had to work on python platforms for some clients that had a dev one day that thought it'd be a good idea to do their saas with python...), and endless infinity of tears of blood and why oh but why???!!!

While it is absolutely possible to do pretty much everything in any language, they still have a domain in which they shine the most.
I feel that Django is a bad choice, sorry.

I'll see what I can do to help. You're a courageous one. Next time, please, oh please, just drop django altogether :'(

I agree that Python/Django is not an ideal platform, based on my experiences trying to maintain this stuff. However, it makes sense as a "next iteration" thing and I seriously appreciate Mindiell taking on the task of updating it! Translating the current code from very old Django to newer, better Django is tedious and thankless work that no one else wanted to do, and Mindiell seems to have done it!

I had been working in fits and starts on a NoSQL idea, hoping to leverage the replication features in order to let users have much fuller access to the DB. But I have a non-computer related job now, and it's hard to find time to work on projects like that. Still, a very interesting idea IMO.

At the moment, it's very hard to fix bugs or add features to the Necta code due to the age of the platform and libraries involved. Even trying to fend off the AI-bot attack in the last few days was difficult, as the external/contrib libs Necta uses for auth, etc are by now so old that they're effectively undocumented.

A rewrite into newer Python/Django platform will allow us to solve some of the frustrating issues that have been building up. I'm hoping that we can imagine something more "outside the box" once we have a stable Necta functioning on actual modern code.

I know but we might aswell start the next version while the current in progress finishes. There might be things that we do bit by bit.
For example, if the new version has a rest api, we could easily keep it as an external service for some parts until we migrate everything.
Of course helping Mindiell is also important to finish the current "in progress" version.
One is not incompatible with the other.
But we don't want to have Mindiell carry this weight alone and have to bugfix everything by himself later on.
If we want to have something more "living" we need lnagugages everybody can easily contribute to.

Again, I salute the effort done, I understand the context, and I can only imagine the amount of dedicated work Mindiell put in this.
I just propose that we try migrating to a more accessible and known stack so that everybody can propose features and collaborate together. This would also avoid having only one or two people spending their weekends fixing bugs all alone.

I'll go check the code again later.
Mindiell, don't bother splitting the code, we'll try improving this in the next stack. If you feel your version is almost done, let's try to focus on launching it and start again with a new stack where more people can easily contribute.
(With Docker, for example, so that everybody can run the stack easily on their computers).

Nyingen, were can we find a dump of the database (minus logins and passwords ^^) ? The rest is pretty much public right?
How big is the current db? What server is it? Mysql ? Postgre ? Something else ?

Well, Python/Django is used for instagram, youtube, pinterest, spotify just to tell a few.

I'm used to it and demovibes has moved from php to python/django. I don't know if this is the right choice, but as mirrorbird said once : "I believe the main block to "rewriting Nectarine" is not skill, or expertise, but the fact that everybody has got time to start the project, and nobody has got time to finish it.
An annoying problem is that I *know* there are at least two unfinished "Necta remakes" out there..."

So, for now, as a lonely web coder I did upgrade source code as I could and I think it can help the community. If you want to go in a *completely new* and *shining* CI/CD kubernets/docker devops nodejs/vuejs/react/whatever thing, you can, of course.

But demovibes is huge and has a lot of features. This is why, after I tried myself to recode from scratch, I strongly believe this is not the good solution (for now, at least).

I'll continue to work on the Django/Python stuff as I think (but maybe I'm wrong) that it's clearly a vey good framework and language, with a huge community and great libs. But I won't go against actual admins who spent their times fixing everything for real here. I'm just a tool.

Demovibes is a living machine which will live after me

The actual version I have isnot usable for now as external parts are not provided (streams), but I'm sure we can build something from it.

Yep, I know, it's historical choices and huge companies that can afford the extra cost though Youtube use python mostly for scripting and deployment. I'm not here to argue though, I'm pretty sure we could find a bunch of giants using just about any tech.
The main question for us should be about finding contributors, onboarding curve, cost efficiency

Again, I praise the effort and I am not complaining in any way regarding your choice Mindiell, it's a logical one.
I'll see if I can help with the Python Scripts, but if you guys want to be able to maintain the platform add features without spending too much time on it, without it becoming a fixing bug duty, we might aswell try to find a way to make collaboration easier in the future.

It does not mean changing everything either, some services could be kept in python if they make sense.

I am only talking about theory and would certainly not force anything. In the end, it's up to you, it's your call.
I just try pointing out that if you want help, it might be worth considering shifting to a more widely used tech stack.
About 12-15 years ago, everybody thought PHP would die because of PHP 6, so moving to other tech would make sense, and nodejs was not ready yet.
The environment has changed yet again.
Check this, you might consider it the likelihood of finding someone that can help
https://w3techs.com/technologies/overview/programming_language

In any case, just trying to help, suggest things, and I'll see if I can support a bit of coding with my rusty Python skills.
You make the choice you want, Nectarine is yours only afterall

Somehow I get "the ick" hearing about enterprise development practices in relation to a scene site.

I don't mean that it has to be some awful hack, but I feel that a WELL-COMMENTED and DOCUMENTED production by a small team would work well. It's not necessary to do all this unit-testing and black-boxing and fucking UML diagrams. We are not pitching a radio plan to Delta Airlines. Demosceners are hackers: they make something small and simple that works.

I wonder why Django was used in the first place, really: look at the agony of searching when every single result has to go through some kind of framework process. Perhaps it came with simple enough CRUD out of the box that it was seen as a time-saver, when there were only 1000 tunes to sort through. But the good, fast external search everybody uses (which ers35 put together overnight, I imagine) is exactly what you'd hack in the first place: taking a quickly accessible data set or flat file, and parsing that. Not calling an instance method on it.

something something impedance mismatch.

As far as the risk of obsolescence: the general Web dev world is much worse for that (look at the number of stupid JavaScript frameworks that appear and disappear every month, like quantum bubbles). Plus any discovered vulnerability applies across every site using the framework. I'm on Team Homebrew (WITH BACKUPS).

Furthermore: the demoscene pre-dated the popularity of 'open source'. It was just making stuff for fun. Swappy the floppy.

Ishaan has been talking about actually taking money to develop stuff. If that ends up being the choice, then I think we should choose NOT to develop things, rather than develop things for money. It's just not scene. It is better to wait five years for some nerd to do it as a hobby, than to pay (!) someone to — I mean — why did anyone ever code a demo? Certainly not for profit.

And the suggestion that "some language may get out of date" — yes, yes, it happens to us all — but I would rather have it repaired by demo nerds than by an enterprise support contract. You can get fucked either way, I would prefer that we get fucked in our own (EXTRA-LEGAL) way.

I don't mean to bitch but I really dislike the way that this conversation has been going. It feels like Coca-Cola sliding into your cool local pub and saying "well, we could do it properly". Yes but then it would be Coke.