Nectarine

Hey there, I see that Ishaan is intereseted in helping moving demovibes source code forward

Is there a possibility to put something on a test/preprod server in order to be able to test new source code with real data ? Maybe with the help of communicator ?

What do you think about this nyingen ?

I don't want to move too fast because data migration can be headhach. For now I never modified database structure from the original demovibes repo.

I think that should be possible; there is a dev server already that has a (slightly out of date?) copy of the database. Getting a necta beta set up on that server shouldn't be a problem, I imagine.

Depending on the db structure, it might not be necessary to modify it. It might be good enough.
However, if the structure is bad, the first step would be to redesign the business logic, the "flow" of the app, and to map models on this flow. Then we'll structure the data to serve the business flow (and optimize it, if needed, to improve resources management and efficiency on the db). This mean we could temporarily rely on orm to save dev time. An orm that work with db mapper pattern instead of active record so we have more flexibility with data structure.

It'll always be time to migrate data (wherever it is required) using a command line script or import mechanism usong data mappers.

And big files are a problem in the sense that you'll have a lot of coupling AND a lot of painful refactoring coming the moment you want to split properly.

The idea here is to bring constructive feedback, not to criticize for the sake of it.
Ideally, the code would be in a language and using framework that are widely used and mastered so mote people could add pull requests and contribute.
Having an open ticket system were people could take a ticket and propose code would be great.

Regarding infrastructure, I could ask a devops in my team to work on this too to propose something, if needed.
Asking one or two people to handle the whole thing is too painful.
We'd need to lake it an open sourced project, with a set of qualified reviewers (aka professional of the field and the server owners playing the role of product owners)

That's great

Code is open and available here : https://framagit.org/demovibes/demovibes
I already created some issues, for migration or enhancement.

For data structure (database), I didn't touch anything not because it was good or bad but because migration will be easier. The main problem was that demovibes source code was using an obsolete version of Django and python2).

Ideally, being able to have a server where code could be automatically deployed would be sufficient I think.

Actual demovibes production source code has been modified by nyingen (and maybe others ?) to add some features and correct some bugs. We'll need to get this part of code to be able to push everything on production.

So :
1. I upgraded lot of stuff and simplified some
2. I think it's time to put it somewhere to get help (being alone can be painfull)
3. With the help of nyingen (and others), we'll be able to upgrade and protect this part of internet and demoscene

EDIT: And clearly, actual source code is not sufficient because I worked only on the website part, not the stream part. And without sufficient datas I'm not able to test it correctly for now.

Mindiell, my only concern now is the choice of Python and Django to be honest.
We should have moved away from both for a revamp as they have a small developer base and are not the optimal choice (does not matter how much one likes python, it is better suited for AI now than for pure web dev).
I am worried that putting effort in another django version is only shooting ourselves in the foot as it'll be another version that will never be updated again as nobody will be willing to put in the effort.
Had we had nodejs (any framework of your choice), or PHP (Symfony for example), find contributors would be piece of cake. Find tools for dev would be piece of cake... Even Go would have felt more suitable for the backend coupled with any front framework...

While I would not hesitate to use python for AI, or blockchain, or data pipelines, I feel that it is not the best suited for web applications...
And so far, practical experience has concluded the same (had to work on python platforms for some clients that had a dev one day that thought it'd be a good idea to do their saas with python...), and endless infinity of tears of blood and why oh but why???!!!

While it is absolutely possible to do pretty much everything in any language, they still have a domain in which they shine the most.
I feel that Django is a bad choice, sorry.

I'll see what I can do to help. You're a courageous one. Next time, please, oh please, just drop django altogether :'(

I agree that Python/Django is not an ideal platform, based on my experiences trying to maintain this stuff. However, it makes sense as a "next iteration" thing and I seriously appreciate Mindiell taking on the task of updating it! Translating the current code from very old Django to newer, better Django is tedious and thankless work that no one else wanted to do, and Mindiell seems to have done it!

I had been working in fits and starts on a NoSQL idea, hoping to leverage the replication features in order to let users have much fuller access to the DB. But I have a non-computer related job now, and it's hard to find time to work on projects like that. Still, a very interesting idea IMO.

At the moment, it's very hard to fix bugs or add features to the Necta code due to the age of the platform and libraries involved. Even trying to fend off the AI-bot attack in the last few days was difficult, as the external/contrib libs Necta uses for auth, etc are by now so old that they're effectively undocumented.

A rewrite into newer Python/Django platform will allow us to solve some of the frustrating issues that have been building up. I'm hoping that we can imagine something more "outside the box" once we have a stable Necta functioning on actual modern code.

I know but we might aswell start the next version while the current in progress finishes. There might be things that we do bit by bit.
For example, if the new version has a rest api, we could easily keep it as an external service for some parts until we migrate everything.
Of course helping Mindiell is also important to finish the current "in progress" version.
One is not incompatible with the other.
But we don't want to have Mindiell carry this weight alone and have to bugfix everything by himself later on.
If we want to have something more "living" we need lnagugages everybody can easily contribute to.

Again, I salute the effort done, I understand the context, and I can only imagine the amount of dedicated work Mindiell put in this.
I just propose that we try migrating to a more accessible and known stack so that everybody can propose features and collaborate together. This would also avoid having only one or two people spending their weekends fixing bugs all alone.

I'll go check the code again later.
Mindiell, don't bother splitting the code, we'll try improving this in the next stack. If you feel your version is almost done, let's try to focus on launching it and start again with a new stack where more people can easily contribute.
(With Docker, for example, so that everybody can run the stack easily on their computers).

Nyingen, were can we find a dump of the database (minus logins and passwords ^^) ? The rest is pretty much public right?
How big is the current db? What server is it? Mysql ? Postgre ? Something else ?

Well, Python/Django is used for instagram, youtube, pinterest, spotify just to tell a few.

I'm used to it and demovibes has moved from php to python/django. I don't know if this is the right choice, but as mirrorbird said once : "I believe the main block to "rewriting Nectarine" is not skill, or expertise, but the fact that everybody has got time to start the project, and nobody has got time to finish it.
An annoying problem is that I *know* there are at least two unfinished "Necta remakes" out there..."

So, for now, as a lonely web coder I did upgrade source code as I could and I think it can help the community. If you want to go in a *completely new* and *shining* CI/CD kubernets/docker devops nodejs/vuejs/react/whatever thing, you can, of course.

But demovibes is huge and has a lot of features. This is why, after I tried myself to recode from scratch, I strongly believe this is not the good solution (for now, at least).

I'll continue to work on the Django/Python stuff as I think (but maybe I'm wrong) that it's clearly a vey good framework and language, with a huge community and great libs. But I won't go against actual admins who spent their times fixing everything for real here. I'm just a tool.

Demovibes is a living machine which will live after me

The actual version I have isnot usable for now as external parts are not provided (streams), but I'm sure we can build something from it.

Yep, I know, it's historical choices and huge companies that can afford the extra cost though Youtube use python mostly for scripting and deployment. I'm not here to argue though, I'm pretty sure we could find a bunch of giants using just about any tech.
The main question for us should be about finding contributors, onboarding curve, cost efficiency

Again, I praise the effort and I am not complaining in any way regarding your choice Mindiell, it's a logical one.
I'll see if I can help with the Python Scripts, but if you guys want to be able to maintain the platform add features without spending too much time on it, without it becoming a fixing bug duty, we might aswell try to find a way to make collaboration easier in the future.

It does not mean changing everything either, some services could be kept in python if they make sense.

I am only talking about theory and would certainly not force anything. In the end, it's up to you, it's your call.
I just try pointing out that if you want help, it might be worth considering shifting to a more widely used tech stack.
About 12-15 years ago, everybody thought PHP would die because of PHP 6, so moving to other tech would make sense, and nodejs was not ready yet.
The environment has changed yet again.
Check this, you might consider it the likelihood of finding someone that can help
https://w3techs.com/technologies/overview/programming_language

In any case, just trying to help, suggest things, and I'll see if I can support a bit of coding with my rusty Python skills.
You make the choice you want, Nectarine is yours only afterall

Somehow I get "the ick" hearing about enterprise development practices in relation to a scene site.

I don't mean that it has to be some awful hack, but I feel that a WELL-COMMENTED and DOCUMENTED production by a small team would work well. It's not necessary to do all this unit-testing and black-boxing and fucking UML diagrams. We are not pitching a radio plan to Delta Airlines. Demosceners are hackers: they make something small and simple that works.

I wonder why Django was used in the first place, really: look at the agony of searching when every single result has to go through some kind of framework process. Perhaps it came with simple enough CRUD out of the box that it was seen as a time-saver, when there were only 1000 tunes to sort through. But the good, fast external search everybody uses (which ers35 put together overnight, I imagine) is exactly what you'd hack in the first place: taking a quickly accessible data set or flat file, and parsing that. Not calling an instance method on it.

something something impedance mismatch.

As far as the risk of obsolescence: the general Web dev world is much worse for that (look at the number of stupid JavaScript frameworks that appear and disappear every month, like quantum bubbles). Plus any discovered vulnerability applies across every site using the framework. I'm on Team Homebrew (WITH BACKUPS).

Furthermore: the demoscene pre-dated the popularity of 'open source'. It was just making stuff for fun. Swappy the floppy.

Ishaan has been talking about actually taking money to develop stuff. If that ends up being the choice, then I think we should choose NOT to develop things, rather than develop things for money. It's just not scene. It is better to wait five years for some nerd to do it as a hobby, than to pay (!) someone to — I mean — why did anyone ever code a demo? Certainly not for profit.

And the suggestion that "some language may get out of date" — yes, yes, it happens to us all — but I would rather have it repaired by demo nerds than by an enterprise support contract. You can get fucked either way, I would prefer that we get fucked in our own (EXTRA-LEGAL) way.

I don't mean to bitch but I really dislike the way that this conversation has been going. It feels like Coca-Cola sliding into your cool local pub and saying "well, we could do it properly". Yes but then it would be Coke.

Mirrorbird: I just want to clarify something, which is not exacl here.

> Ishaan has been talking about actually taking money to develop stuff.

Yes and no. And mostly No actually.
I said, in substance, that I was ready to spend free time, for free to help and support dev.
I said that in this context, of course, I would do it for free.
And, as a side joke (because I perfectly know this would never happen), I said that if the team wanted to hire a full team to do it with UX/UI Designers, developers, etc, then I said that this could no longer be free.

I feel sorry that you took a shortcut on this as it is absolutely NOT a representation of my initial intention. Please read again carefully and without taking shortcuts.

NOW, with all that said
It's all nice and such to rely of "nerds" spending their free time. That's how most projects disappear by the way. You might prefer Nectarine to die and disappear for the sake of your own ideology, please allow us to have hope in its continuation.
You also seem to say that enterprise practices are wrong and that things should be hacked. But that's not how long lasting projects work.
Demoscene is all about one-shots, make it, show it one time, trash it. There reason nectarine is buggy and you have a laggy sucky search, bugs everywhere is exactly this.
Using proper practices tomake something sustainable (not talking about money), enjoyable, is OK.
Website does not have to be shitty just for the sake of it being "hacker wanabee like".
Demoscene is not only about hacking quick and dirty, in my opinion it was also about contest and striving for excellence. You are showing what you are capable of within constraints. Limited time and resources make it a one off philosophy but in regard to that spirit of excellence, given time and manpower, shouldn't we strive for something great?

Anyway, I repeat what I offered because I absolutely hate when people report things I "said" in a way that is full of shit, AND to make sure it is crystal clear:

- I offered my help on my free time and for free. Several times.
- I offered to bring practices from the pro world to help make something that can last (and evolve with everybody being able to contribute).
- I tried to bring proper arguments on the table (and tried to avoid trolling) and proper action plan to HELP. I never said to stop everything and throw away everything.
- I repeatedly said that I would NOT force anything, that I was not one to decide, that you guys were the one in control of things.
- I stated that I felt that Django shouyld probably be avoided for future version, especially because it makes it difficult for people to contribute and put too much pressure of a very few people sacrificing their free time for the rest.

Yet you still find a way to misinterpret things and be full of shit here (sorry but I feel frustrated by how you misrepresent me, I have no personal vendetta against anyone here).
You claim it's best to put the pressure on 1 or 2 people, that you'd rather have them support things out, and throw a sucker-punch complaining about the quality of the site saying some other people randomly arranged a search in a few hours spreading shit on the effortts that were done without bringing ANY help on the topic.

Sorry about that but it's painful to read. It should be hacked and be done by random nerds, you hate pro world bringing things on the table, yet you complain about the problems inherents to this "hacking things" development type.
The current website was certainly NOT done with UML, best practices etc. Otherwise we would probably not have these many obvious problems. And aould not have had them for a decade.

Now with all that said, I believe that I have been offering free help, given my point of view regarding choices, proposed possible course of action as a way to approach this.
At least I have tried contributing.
Not just complaining.

You are anchored in a dream of the past that will never happen again. This era of "back then" is dead. And most of these hacker developers are working as pros in the industry anyway.

Sorry, my last sentence was uncalled for and out of frustration, I should have been able to disregard this. Just could not.

Mindiell, I also sincerely hope I did not offend you. I only came with the intention to help and bring constructive ideas, I also mentioned I'd gladly accept the fact they may not be welcomed.
Désolé si je t'ai vexé, ce n'était pas du tout mon intention. J'espère que ce n'est aps le cas, encore une fois, je ne peux que féliciter la prise d'initiative et l'effort, je comprends le contexte et le choix, et je n'étais pas venu pour troller mais proposer une feuille de route qui ait du sens pour la suite. Bref, je ne sais pas trop pourquoi cette peur de faire des choses un pe plus encadrées, mon idée étant simplement que tu ne te retrouves pas à porter le fardeau seul ensuite.

Seriously ? I can't be offended.

I'll continue to help one the Django version of demovibes because this is the easiest way. If, at some point, someone want to rebuild everything, I won't block him/her.

If nobody wants to help : no problemo, I'll continue on my long way alone ;o)