Updates and upgrades for 2.0

The situation you describe is sadly a universal one, and not only in open source. The way I see it, is that ImpressCMS can only take responsibility for the core and the core modules when it promises an upgrade path. We cannot take responsibility for all the other modules that are out there, whether they are still supported or not.

Your point about what it takes to make a module 2.0-compatible is a good one. I understand that Mekdrop made every effort to keep backward compatibility with the old modules, but those old modules will only offer the old ways of working. Once we have a more stable 2.0, the documentation should be made available, firstly on how to create new modules from scratch, and after that how to make existing modules compatible.

Keep in mind that we are bridging multiple gaps here : ImpressCMS 1.4 -> ImpressCMS 2.0, but also PHP 7 -> PHP 8.x. It is very well possible that modules are simply not compatible with the new PHP version, before looking at the ImpressCMS version.

We will need to do extensive tests on upgrading module versions, and define a general upgrade policy.

What are the intermediary steps for someone who has created or maintains a module to prepare it for compatibility with upcoming releases of the core? A module doesn't work without the core, and the core alone isn't enough to create a website. We need them to progress in parallel.

With the changes in the later releases of 1.4, and the removal of deprecated functions, files, and folders in 1.5, the original scope for 2.0 has been realized, at least based on this (and others) - Navigating ahead - ImpressCMS Compass 2012. Maybe we should relabel 1.5 as 2.0 and what is in 2.0 as 3.0.

Is the information in the debug panel enough to help get a module ready for what's next?

What are the intermediary steps for someone who has created or maintains a module to prepare it for compatibility with upcoming releases of the core?

Normally, modules that do not use removed or deprecated functions should continue to work as they work now. I haven't been able to test that for the moment, as I have issues setting up a 2.0 version currently.

Using 2.0 functionality is not backward compatible, and @Mekdrop was looking into adapting imBuilding so that it generates first-class citizens for ImpressCMS 2.0.

Maybe we should relabel 1.5 as 2.0 and what is in 2.0 as 3.0.

This action would take care of the number only. That version would still lack several fundamental parts that a current-day CMS is expected to provide, and that are in the 2.x branch currently. The need for 3.0 would be as pressing from the start as the need for 2.0 now Things have changed quite a bit in 10 years.

On the other hand, renumbering 2.0 to 3.0 is just a git branch rename, so it's not really a big deal, it's just version semantics at that point.

For 1.5 I think the main issue is not removing functions from the core - because they are rarely used - but differences in the newer PHP versions.

For 2.0 also the biggest issue is upgrading templates to the newer Smarty syntax. 

I hope that 2.0 will have a modules testing framework, so it will be easier to find if a module breaks with newer versions of CMS or PHP.

----

About that post from 2012 - what I really know is that probably we still will not have the system module rewritten fully with IPF. Sadly I don't have so much time to such big changes.

The 2 points I'm encountering while testing 1.5 currently is an issue with seemingly non-declared constants when installing under PHP 8.0, and deprecated filter types when installing under PHP 8.1 (I think I don't get to the point where the non-declared constant is used, the filter types throw me out sooner ). If those 2 points can be fixed, I think we could consider moving what is currently identified as 1.5 directly to the 2.0 release.

And, as a matter of fact, we should think of writing a new ImpressCMS compass, to identify the direction of the project and put it somewhere explicitly. We all have good ideas and the best intentions, but our efforts lack a common direction

@fiammybe I'll try to look into these issues at the end of the week.

Super news! I tried changing the deprecated filter typ with a 'default' filter, but that broke everything (white page) so the solution isn't as simple as that.

Thank you both for contributing to every release of ImpressCMS! We all have a stake in continuing and moving forward.

My personal interest is continuity for sites I maintain and the businesses they support. For the most part, they aren't needing any new functionality. The sites aren't the products of these businesses. The sites provide gateways to the businesses through their content, through the search results they are earning. The primary need of these sites is continuity - their ability to exist in the change ecosphere of the web. As a site maintainer, the main thing I need is the least amount of friction in keeping the sites running.

To maintain continuity, being able to run on the currently supported versions of PHP and MySQL are the primary factors. Doing this also improves performance, another critical component of a viable website supporting an organization, a project, or a business. I can commit to this - helping updating the core and key modules to be compliant with PHP 7 and 8. I believe we can do this without changing the minimum requirements for PHP and MySQL dramatically.

When we couple this with the commitments we originally made for the next major release of ImpressCMS, we can have a major impact on our constituents. First - we did what we said we were going to do. Second - we deliver something that performs better, runs on current web servers, and displays on all types of devices. That's what keeps me up at night.

Thanks for clarifying that Steve! 

My situation is a bit different : my websites are standalone, and not linked to businesses at the moment. I am part of ImpressCMS in order to evolve and improve the experience by adding features, reworking existing features so they can be improved, and also removing features if they are no longer relevant for the future direction we want to go in. That last can be a difficult decision, but in those cases you need an alternative of course.

I think the way @Mekdrop and I want to evolve with ImpressCMS is not always compatible with the more-or-less status quo you desire : we already did major changes in the architecture to be able to evolve in a certain direction that will make it difficult to offer a one-click migration from an existing older site. I'll see with Mekdrop what are the best ways of doing that, but we might need a few manual steps to migrate from one module to another (something that might benefit the end-user in any case, the possibility to move data between modules).

I'm glad you want to focus your efforts on keeping the lights running with the current codebase, that means I will be able to focus a bit more on getting the next generation ready for primetime together with Mekdrop. It's really time we got that one in the wild.

After thinking a bit more on this, perhaps it would be beneficial to split off the 2.x branch in a separate github codebase. It would make life much easier for everyone I think when we no longer have 2 versions running in parallel in the same github project.

I thought about that a bit and I think maybe for now we could just make 1.5.x default branch. 

Still, I think that 1.x days are numbered just because some PHP libs that are used core (f.e. Smarty) don't have non-breaking upgrades available.

And then comes another question how to make sure that all modules are working correctly after these upgrades? I here only two possible solutions: a) try to grow our community that would test everything for us b) write some tests and use CI/CD to test all needed variants. Sadly modules with ICMS 1.x are not easily tested with (b) variant and for (a) variant we are still struggling. 

Also because it's hard for CI/CD to be implemented for testing modules with ICMS 1.x more manpower is needed to keep up with some required libs for modules upgrades. We can't use Dependabot and still make sure that everything runs correctly.

Maybe there are some other ways how to achieve that but right now I don't see any. 

4th attempt to reply from my phone. Not a good user experience. Blank pages, getting blocked by Protector, and then not knowing how I got sent back to reading and not replying. This is what we really have to address. At least CKEditor works on mobile devices. 
Regarding CI - @jegelstaff implemented it in Formulize and ImpressCMS 1.0. It can be done.

I agree that can be done but I think is much harder.

From what I see @jegestaff added only automated tests. Idk how much time would need to require to add add also automatic dependencies upgrading and other time-saving tasks.

Yes, maybe there is a way but at least for now I don't see something that would look easy.

In any case @skenow if you wish to maintain 1.x I don't mind but I just don't see any way how to keep up always with the required updates (run on newer PHP/MySQL and be secure) and that code.

For the time being, nothing changes to the github setup, even though it may be a bit confusing. The 1.5.x branch is being prepped to work under PHP 8.0. From the number of changes that need to be done, we accumulated quite a bit of technical debt there.

@skenow asked the question if we shouldn't just release the new version with PHP 7.4 support only, but I have some issues with officially releasing a version for an already deprecated PHP version

For those of you that are adventurous : the current codebase works perfectly well under PHP 7.4

Things I'm currently doing is upgrading the external libraries. Some of them (like HTMLPurifier) need a version upgrade to be PHP8 compatible.

Granted, this 8.0 requirement is turning out to be much more than anticipated, but I'm confident that it is a step that we need to take. It will give us better code overall, which is always a good thing stability-wise.

PHP7.4 is deprecated - that doesn't mean every site and server are running at that version. How many sites are still running under PHP5.6 or less? Even getting to that version could be an adventure. I've found a number of ImpressCMS sites that will have to take extra steps before they upgrade, some of mine are included in this list.

At a minimum, they will have to upload the new version and switch their server to the new version of PHP in order to complete the upgrade. If they haven't switched to PDO, they will have to do that, too, before they proceed. I use Formulize which implemented PDO independent of the core and activating PDO for the site causes problems with the module.

These changes are to be expected - I feel we need to be very open about what needs to happen and help them anticipate and navigate the requirements.

I've been thinking about a pre-upgrade script that could be uploaded to their site, or added as a custom block, that will do check the preliminaries a little more closely. We can't really do this in the current system block because they would have to upgrade to get the added tests. It could also scan for the use of deprecated functions and methods that were removed in 1.4 and the upcoming 1.5. The deprecated messages showing when debug is turned on is also a good place to make sure you're ready to move ahead.

According to PHP Code Sniffer and the PHP Compatibility rule, our current 1.5.x branch will have errors below PHP5.6. The changes to make ImpressCMS run on PHP7+ haven't raised the minimum required version as of this point.

There still are some things that need to be addressed to eliminate errors when running under PHP7.0 - PHP7.4. Some are external libraries, as @fiammybe noted, and some are in code that won't get used (mysql extenstions aren't used when PDO is the db connection type, but the code is still there) or get used often (mostly in external libraries).

We do need to become PHP8+ compliant ASAP. Will having another release come sooner get us more testing with the various environments everyone is using? Will it help get other modules updated to be compliant with the newer versions of PHP faster?

Once we have a proper scope, we can better estimate the timeline to deliver.

This all depends on requirements and how far we want to go here. Getting this release out sooner, while it is not yet fully 8.0 compliant could be an option. However once that branch is out, I don't intend to focus much more time on it, so further releases to polish up this branch would be mostly up to you then. This renumbering has had too much of an impact already on the development of the next version (former 2.0 branch).

If I spend time on modules, it will first of all be to rewrite them to be compliant with the next version. I might do bugfixes for the older version, but there are so many modules that need attention that I cannot commit to php version upgrades or feature improvements.

I would expect you could say that 2.0 will only upgrade from 1.4.4, partly because of the difficulty to get everything to run on such a wide range of PHP versions. But I'm not in that situation that I need to upgrade from sites older than 1.4.2.

I think the answer to the question about PHP compatibility and icms is same as 'what modules should run on newer PHP?'. Because there is no quick way to fix everything. So at least for now ICMS should run at least something that is required by most users.

You're correct there @Mekdrop. And that is the difficulty : we have no idea what situation most of our users are in. I think Skenow is one of the user with most active sites a the moment, and I know of a few users in Germany that are awaiting newer PHP support because of the shutdown of PHP 7.4 by hosting companies there (they try to be up-to-date).

But other than that, it's a big unknown.

I'm trying to get htmlpurifier 4.15.0 standalone updated but Git is giving me some dificulties with unchanged files that it keeps marking as changed. And I'm still waiting for the PRs for 1.5.x to be approved in order to be able to continue.

I made a new milestone for this release called new 2.0

This will make it easier to follow up what should be finalised before releasing the final release.