Skip to main content


Query optimizations


!Friendica Developers

Hi,

Since I'm running one of the larger nodes (opensocial.at, friendica.me), I'm constantly looking for bottlenecks and improvements.

I log all slow queries for later analysis. Would it be helpful to post it anywhere with some insights of the database (how much data is in it, how many entries post-user or contact do have, ...)

For example this query:
SELECT `circle`.`id`, `circle`.`name`, (SELECT COUNT(*) FROM `post-user` WHERE `uid` = 1137 AND `unseen` AND `contact-id` IN (SELECT `contact-id` FROM `group_member` AS `circle_member` WHERE `circle_member`.`gid` = `circle`.`id`) ) AS `count` FROM `group`
AS `circle` WHERE `circle`.`uid` = 1137
takes around 17 seconds and is constantly logged in the slow-query logs.

Shall I post it here, is it even useful? shall I open an issue for such cases? Is it just a problem for the node, so I post it at Friendica Admins for help?

@Michael Vogel what do you think :)

Friendica Developers reshared this.

in reply to Philipp Holzer

There is a setting where you can disable the counts for unseen posts in circles. I guess that we have to change something fundamental there, means not only a better query, but some changes changes to the data model.


Slow /photo/preview performance


!Friendica Admins

Hi all,

Because of the ongoing performance issues, I moved my whole Friendica database of opensocial.at and friendica.me to a dedicated root-server just for mariadb with 128GB RAM.

At first, it seemed like it helped and opensocial.at is faster than before (the performance of friendica.me is still a mess, but I will analyze it later), but the /photo/preview route is now going crazy:

It seems like the RAM I/O issue I had (which is now gone) is now replaced by an Network throughput issue - but at least the network I/O seems fine:

Do you have any idea what I can do now?

One thing would be to add a "StorageClass" for the Photo to load them (directly) from a filedirectory/block storage instead of the database.

Friendica Admins reshared this.

in reply to Philipp Holzer

@Philipp Holzer Have you checked your /tmp folder? Here on my server it ran full with magick-XXXXX files.
in reply to Roland Häder

Thanks for the hint
But there are just two zero-byte magick files at /tmp :)

Friendica Admins reshared this.



CI currenrly not usable


!Friendica Developers

Hi, I recently tried to increase the woodpecker / ci performance. Unfortunately it dropped the repositories after migrating from sqlite to MySQL as backend. @Tobias re-added it again (thanks :-) ) but now some settings are still missing.

So currently, the CI isn't working :-( I'll be back home at Sunday evening, but I think I will fix it not before Monday/Tuesday..

So if you need to merge new PRs, keep in mind that the files under files.friendi.ca aren't updating ...

Sorry and I'll give an update when it's working again..

Friendica Developers reshared this.



Geplante Downtime / Server Migration


!Opensocial.at
-- English below

Liebe opensocial.at BenutzerInnen,

Ich starte nun eine Migration von opensocial.at auf eine stärkere server-hardware. Dadurch entstehen temporäre Downtimes in den kommenden Stunden.

Ich möchte mich für die Unannehmlichkeiten entschuldigen, aber ich denke, dass dadurch die gesamte Performance von opensocial.at wieder besser werden sollte :).

Lg,
Philipp

----

Dear opensocial.at user,

I'm planning to migrate my server infrastructure onto a more powerful environment.
The goal is to increase the performance, as, I think, you all currently experience a downgrading performance.

I hope it will help :)

It will take some hours, where opensocial.at will be temporary unavailable from time to time.

Regards,
Philipp

Opensocial.at reshared this.

in reply to Philipp Holzer

Leider hat die Migration nicht geklappt und ich musste wieder zurückrollen.
--> Ich probiere es im August noch einmal

---

Unfortunately, the migration didn't work so I had to rollback.
--> I will retry again in august

Opensocial.at reshared this.



Planned Downtime / Migration


!friendica.me

Dear friendica.me user,

I'm planning to migrate my server infrastructure onto a more powerful environment.
The goal is to increase the performance, as, I think, you all currently experience a downgrading performance.

I hope it will help :)

It will take some hours, where friendica.me will be temporary unavailable from time to time.

Regards,
Philipp

friendica.me reshared this.

in reply to Philipp Holzer

Unfortunately, the migration didn't work so I had to rollback.
--> I will retry again in august

friendica.me reshared this.



@Der Motzmichel :mastodon: - ich habe deinen Account bei opensocial.at abgelehnt, weil er am selben Server läuft wie Friendica.me , mit dem du bereits schon nicht glücklich wurdest.. probier libranet.de




#FediBlock *.mastotroll.netz.org

Admin (of friendica.utzer.de) - 2023-05-03 18:16:56 GMT

New troll domain with many subsomains. This is a strongly recommended block for all #Fediverse services for *.mastotroll.netz.org.Please share this post!

#Friendica @Friendica Admins
cc @utzer [Friendica]





Slow photo requests / inbox requests


Hi !Friendica Admins ,

I'm currently tracing down performance issues, I' currently suffering from on my nodes opensocial.at and friendica.me .

It seems like the main performance problems are

  • delivering photos
  • [code]/inbox/code] requests


You can see some peeks and some long runners here:

The long runners (10sec) are the /inbox requests, the peaks are delivering photos after opening conversations, network pane, ...

Do you have any hints how I could improve these two types of requests?

The database is currently using ~50GB RAM, I set them as high as possible.

Friendica Admins reshared this.

Unknown parent

friendica (DFRN) - Link to source
Philipp Holzer
currently, "movetoavatarcache" is running on my node :-) 17000 from 59000 finished ;)

Friendica Admins reshared this.

in reply to Philipp Holzer

Hi, after moving all contact images to avatar/, there are still a lot of long running requests, see the image above. Do you have any idea? The overall CPU/RAM is on a normal state.

Friendica Admins reshared this.


Philipp Holzer reshared this.


Friendica 2023.04 released


We are very happy to announce the availability of the new stable release of Friendica “Giant Rhubarb” 2023.04. The highlights of this release are For details, please the CHANGELOG file in the repository. What is Friendica Friendica is a decentralized comm

We are very happy to announce the availability of the new stable release of Friendica “Giant Rhubarb” 2023.04. The highlights of this release are

  • The Mastodon compatible API of Friendica was improved,
  • Drag and Drop functionality was added to the Frio theme for image attachments, and
  • Christian Pöschl from usd AG reported a XSS fulnurability in a javascript dependency in the gallery of the Frio theme that was fixed.

For details, please the CHANGELOG file in the repository.

What is Friendica


Friendica is a decentralized communications platform, you can use to host your own social media server that integrates with independent social networking platforms (like the Fediverse or Diaspora*) but also some commercial ones like Tumblr.

How to Update

Updating from old Friendica versions


If you are updating from an older version than the 2022.12 release, please first update your Friendica instance to that version as it contained some breaking changes.

Pre-Update Procedures


Ensure that the last backup of your Friendica installation was done recently.

Using Git


Updating from the git repositories should only involve a pull from the Friendica core repository and addons repository, regardless of the branch (stable or develop) you are using. Remember to update the dependencies with composer as well. So, assuming that you are on the stable branch, the commands to update your installation to the 2023.04 release would be
cd friendica
git pull
bin/composer.phar install --no-dev
cd addon
git pull
If you want to use a different branch than the stable one, you need to fetch and checkout the branch before your perform the git pull.

Pulling in the dependencies with composer will show some deprecation warning, we will be working on that in the upcoming release.

Using the Archive Files


If you had downloaded the source files in an archive file (tar.gz) please download the current version of the archive from friendica-full-2023.04.tar.gz (sha256) and friendica-addons 2023.04.tar.gz (sha256)) and unpack it on your local computer.

As many files got deleted or moved around, please upload the unpacked files to a new directory on your server (say friendica_new) and copy over your existing configuration (config/local.config.php and config/addon.config.php) and .htaccess files. Afterwards rename your current Friendica directory (e.g. friendica) to friendica_old and friendica_new to friendica.

The files of the dependencies are included in the archive (make sure you are using the friendica-full-2023.04 archive), so you don’t have to worry about them.

Post Update Tasks


The database update should be applied automatically, but sometimes it gets stuck. If you encounter this, please initiate the DB update manually from the command line by running the script
bin/console dbstructure update
from the base of your Friendica installation. If the output contains any error message, please let us know using the channels mentioned below.

Please note, that some of the changes to the database structure will take some time to be applied, depending on the size of your Friendica database.

Known Issues


During the release candidate phase of this release, we had reports that changes to the Tumblr addon combined with PHP version 7.4 might lead a WSoD situation on some pages and connection problems with Tumblr (see 12979 and 13021). These problems could be solved for this release. The Tumblr addon will receive an overhaul in the upcoming release where these issues will be addressed as well.

How to Contribute


If you want to contribute to the project, you don’t need to have coding experience. There are a number of tasks listed in the issue tracker with the label “Junior Jobs” we think are good for new contributors. But you are by no means limited to these – if you find a solution to a problem (even a new one) please make a pull request at github or let us know in the development forum.

Contribution to Friendica is also not limited to coding. Any contribution to the documentation, the translation or advertisement materials is welcome or reporting a problem. You don’t need to deal with Git(Hub) or Transifex if you don’t like to. Just get in touch with us and we will get the materials to the appropriate places.

Thanks everyone who helped making this release possible and have fun!


friendi.ca/2023/04/23/friendic…

in reply to Friendica News

hey.......i would like to know how to get verified on #Friendica .... if anyone would tell me, that would be cool....


Update JSON LD


!Friendica Developers

@Hypolite Petovan already fixed a current JSON-LD issue with git.friendi.ca/friendica/php-j…

Do we need a 1.1.2 release to upgrade it in the Friendica composer.lock file as well?

The logs of my instances are flooded with this error, that's why I'm asking *g*

Friendica Developers reshared this.



BaseURL components (hostname, SSL policy, urlpath)


!Friendica Developers

I'm currently reducing the whole BaseUrl.php code massively.
Do we really need the ssl_policy, urlpath and hostname separate from the system.url?

I will use for the BaseUrl.php, based on the system.url a "real" UriInterface as $this->url, so we don't need saving the scheme, urlpath and hostname separately anymore. They are just useful for the install process but must not be changed afterwards.

The only thing, which I'm unsure is the ssl_policy, because if someone changes it afterwards in the admin site, all URL in all contacts and photos will get updated. But the question is => is this even allowed? I think this could brick the access over federation because the base-url of each entry isn't right anymore. And it isn't supported when the policy is changed by console.

So I would drop it as well and merge all config entries into the system.url.

Additionally, I will replace the Exception with a "CRITICAL" log entry to avoid a WSOD.

Friendica Developers reshared this.

Unknown parent

friendica - Link to source
Hypolite Petovan
@Tobias I'm not sure what wouldn't work with a self-signed certificate. Unless we are checking the CA for self-requests, but I'm not sure the legacy SSL Policy configuration value is about that.

Friendica Developers reshared this.

Unknown parent

friendica (DFRN) - Link to source
Philipp Holzer
I know and I'm using local Vagrant with https and there is no check at all in the code that would fail here. It works fine :-)

Friendica Developers reshared this.



Friendica.me | Opensocial.at Short Downtime because of DB adjustements


!friendica.me , !Opensocial.at

I do have to restart the Database instances because they are too resource hungry.

There's a short downtime possible.

reshared this


Philipp Holzer reshared this.


Friendica 2023.01 released


Christian Pöschl from usd AG has found another XSS vulnerability in Friendica which is close with this hotfix release of Friendica. In addition some other bugfixes for the distribution of forum postings and improvements to the update process of node infor

Christian Pöschl from usd AG has found another XSS vulnerability in Friendica which is close with this hotfix release of Friendica.

In addition some other bugfixes for the distribution of forum postings and improvements to the update process of node information are included in this release.

For details, please the CHANGELOG file in the repository.

What is Friendica


Friendica is a decentralized communications platform, you can use to host your own social media server that integrates with independent social networking platforms (like the Fediverse or Diaspora*) but also some commercial ones like Twitter.

How to Update

Updating from old Friendica versions


If you are updating from an older version than the 2022.06 release, please first update your Friendica instance to that version as it contained some breaking changes.

Pre-Update Procedures


Ensure that the last backup of your Friendica installation was done recently.

Using Git


Updating from the git repositories should only involve a pull from the Friendica core repository and addons repository, regardless of the branch (stable or develop) you are using. Remember to update the dependencies with composer as well. So, assuming that you are on the stable branch, the commands to update your installation to the 2022.12 release would be
cd friendica
git pull
bin/composer.phar install --no-dev
cd addon
git pull
If you want to use a different branch than the stable one, you need to fetch and checkout the branch before your perform the git pull.

Pulling in the dependencies with composer will show some deprecation warning, we will be working on that in the upcoming release.

Using the Archive Files


If you had downloaded the source files in an archive file (tar.gz) please download the current version of the archive from friendica-full-2023.01.tar.gz (sha256) and friendica-addons 2023.01.tar.gz (sha256)) and unpack it on your local computer.

As many files got deleted or moved around, please upload the unpacked files to a new directory on your server (say friendica_new) and copy over your existing configuration (config/local.config.php and config/addon.config.php) and .htaccess files. Afterwards rename your current Friendica directory (e.g. friendica) to friendica_old and friendica_new to friendica.

The files of the dependencies are included in the archive (make sure you are using the friendica-full-2023.01 archive), so you don’t have to worry about them.

Post Update Tasks


The database update should be applied automatically, but sometimes it gets stuck. If you encounter this, please initiate the DB update manually from the command line by running the script
bin/console dbstructure update
from the base of your Friendica installation. If the output contains any error message, please let us know using the channels mentioned below.

Please note, that some of the changes to the database structure will take some time to be applied, depending on the size of your Friendica database.

Known Issues


Regarding the update process none as of writing.

How to Contribute


If you want to contribute to the project, you don’t need to have coding experience. There are a number of tasks listed in the issue tracker with the label “Junior Jobs” we think are good for new contributors. But you are by no means limited to these – if you find a solution to a problem (even a new one) please make a pull request at github or let us know in the development forum.

Contribution to Friendica is also not limited to coding. Any contribution to the documentation, the translation or advertisement materials is welcome or reporting a problem. You don’t need to deal with Git(Hub) or Transifex if you don’t like to. Just get in touch with us and we will get the materials to the appropriate places.

Thanks everyone who helped making this release possible and have fun!


friendi.ca/2023/01/15/friendic…


Philipp Holzer reshared this.


Friendica 2022.12 released


We are very happy to announce the avail-ability of the new stable version of Friendica. Wrapping up the sprint from the 2022.10 release of Friendica we closed 73 filed issues and had almost 300 pull requests by 19 contributors. A special thanks goes out t

We are very happy to announce the avail-ability of the new stable version of Friendica. Wrapping up the sprint from the 2022.10 release of Friendica we closed 73 filed issues and had almost 300 pull requests by 19 contributors.

A special thanks goes out to Christian Pöschl from usd AG and Matthias Moritz who have found a CSRF- and XSS-attack, that is fixed with this release.

In addition to fixing this bug, the highlights of the changes since the 2022.10 release are

  • The default theme of Friendica (frio) got many improvements and some old themes got deprecated.
  • The calendar saw some improvements and can now be made visible to anonymous visitors.
  • The homepage mentioned on the user profile is now automatically verified via the rel-me backlink.
  • Images attached to a posting are now shown in a grid at the bottom of the posting.
  • A moderation corner was established from the admin panel where a future release will add more moderation tool and bundle them with the current once.


For details, please the CHANGELOG file in the repository.

Since version 2022.06 the lowest required PHP version Friendica needed on the server was raised to PHP 7.3 and PHP 8.0 is supported.

What is Friendica


Friendica is a decentralized communications platform, you can use to host your own social media server that integrates with independent social networking platforms (like the Fediverse or Diaspora*) but also some commercial ones like Twitter.

How to Update

Updating from old Friendica versions


If you are updating from an older version than the 2022.06 release, please first update your Friendica instance to that version as it contained some breaking changes.

Pre-Update Procedures


Ensure that the last backup of your Friendica installation was done recently.

Using Git


Updating from the git repositories should only involve a pull from the Friendica core repository and addons repository, regardless of the branch (stable or develop) you are using. Remember to update the dependencies with composer as well. So, assuming that you are on the stable branch, the commands to update your installation to the 2022.12 release would be
cd friendica
git pull
bin/composer.phar install --no-dev
cd addon
git pull
If you want to use a different branch than the stable one, you need to fetch and checkout the branch before your perform the git pull.

Pulling in the dependencies with composer will show some deprecation warning, we will be working on that in the upcoming release.

Using the Archive Files


If you had downloaded the source files in an archive file (tar.gz) please download the current version of the archive from friendica-full-2022.12.tar.gz (sha256) and friendica-addons 2022.12.tar.gz (sha256)) and unpack it on your local computer.

As many files got deleted or moved around, please upload the unpacked files to a new directory on your server (say friendica_new) and copy over your existing configuration (config/local.config.php and config/addon.config.php) and .htaccess files. Afterwards rename your current Friendica directory (e.g. friendica) to friendica_old and friendica_new to friendica.

The files of the dependencies are included in the archive (make sure you are using the friendica-full-2022.12 archive), so you don’t have to worry about them.

Post Update Tasks


The database update should be applied automatically, but sometimes it gets stuck. If you encounter this, please initiate the DB update manually from the command line by running the script
bin/console dbstructure update
from the base of your Friendica installation. If the output contains any error message, please let us know using the channels mentioned below.

Please note, that some of the changes to the database structure will take some time to be applied, depending on the size of your Friendica database.

Known Issues


Regarding the update process none as of writing.

How to Contribute


If you want to contribute to the project, you don’t need to have coding experience. There are a number of tasks listed in the issue tracker with the label “Junior Jobs” we think are good for new contributors. But you are by no means limited to these – if you find a solution to a problem (even a new one) please make a pull request at github or let us know in the development forum.

Contribution to Friendica is also not limited to coding. Any contribution to the documentation, the translation or advertisement materials is welcome or reporting a problem. You don’t need to deal with Git(Hub) or Transifex if you don’t like to. Just get in touch with us and we will get the materials to the appropriate places.

Thanks everyone who helped making this release possible and have fun!


friendi.ca/2022/12/20/friendic…

Unknown parent

friendica (DFRN) - Link to source
Hypolite Petovan
@Jonathan "Mastodon" Lamothe @utzer ~Friendica~ If you didn't add any changes to your local .htaccess file, yes.


opensocial.at database data seems corrupt


!Friendica Support

It seems like I do have a problem with my opensocial.at database :-( .

The database ran full about a year ago (what a shame, I know :( ...) and I had to recover some of the data (otherwise I had to import a backup about ~12hours ago, I tried to avoid to loose posts for this period of time for my users ...). So I stopped it, started it with innodb_force_recovery=3, repaired it and checked it with mysqlcheck.

Two days ago, I updated the mariadb from 1.10.3 to 1.10.10 and now the problems occur again.

I stopped the MariaDB 1.10.3 container, updated it to 1.10.10 and started it again .. And MariaDB started a "crash recovery". I really don't know why... The crash recovery wasn't successfully (I tried it ~4 times), so I had to add innodb_force_recovery=3 again.

Now the database went up, but everytime I stopped it and started it again, the crash recovery appears again.

So after the instance was up and running, I dumped the whole database with mysqldump into one single *.sql , started a brand new MariaDB 1.10.10 and imported the dump again.

So far so good ..

But... unfortunately, after a restart, the crash recovery appears again. So I'm totally lost, what's now happening..

I noticed during the import that one batch of rows took about 2 hours to complete with the result query affected 0 rows.

here's my customized.cnf, which I'm using (it's a Hetzner root server with 64 GB RAM and 12 CPUs):

[mysqld]
query_cache_size = 0
query_cache_type = 0
performance_schema = ON
join_buffer_size = 140M
innodb_buffer_pool_size = 12G
innodb_log_buffer_size = 31M
innodb_log_file_size = 3G
table_open_cache = 1000
max_connections = 400
wait_timeout = 200
interactive_timeout = 4000

log-bin = mysqld-bin
transaction-isolation = READ-COMMITTED
binlog-format = ROW
skip-innodb-read-only-compressed = ON
innodb_read_only_compressed = OFF
innodb_use_native_aio = OFF

innodb_fast_shutdown=0
innodb_max_dirty_pages_pct=0
innodb_buffer_pool_dump_at_shutdown=1
innodb_buffer_pool_load_at_startup=1

For the new MariaDB instance, I didn't add any customization to avoid any wrong options.


currently, it makes a recovery again:

db_1     | 2022-12-16 10:41:00 0 [Note] InnoDB: Rolled back recovered transaction 371172                                                                                                                                                                                                    
db_1     | 2022-12-16 10:41:11 0 [Note] InnoDB: To roll back: 1 transactions, 7611737 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:26 0 [Note] InnoDB: To roll back: 1 transactions, 7608132 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:41 0 [Note] InnoDB: To roll back: 1 transactions, 7603940 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:46 0 [Note] DDL_LOG: Crash recovery executed 1 entries                                                                                                                                                                                                          
db_1     | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '0.0.0.0'.                                                                                                                                                                                                             
db_1     | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '::'.                                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:47 0 [Note] mariadbd: ready for connections.                                                                                                                                                                                                                    
db_1     | Version: '10.10.2-MariaDB-1:10.10.2+maria~ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution                                                                                                                                               
db_1     | 2022-12-16 10:41:56 0 [Note] InnoDB: To roll back: 1 transactions, 7601453 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:11 0 [Note] InnoDB: To roll back: 1 transactions, 7600146 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:26 0 [Note] InnoDB: To roll back: 1 transactions, 7598395 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:41 0 [Note] InnoDB: To roll back: 1 transactions, 7595967 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:56 0 [Note] InnoDB: To roll back: 1 transactions, 7594000 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:11 0 [Note] InnoDB: To roll back: 1 transactions, 7590646 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:26 0 [Note] InnoDB: To roll back: 1 transactions, 7585618 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:41 0 [Note] InnoDB: To roll back: 1 transactions, 7579929 rows
db_1     | 2022-12-16 10:43:56 0 [Note] InnoDB: To roll back: 1 transactions, 7574058 rows
db_1     | 2022-12-16 10:44:11 0 [Note] InnoDB: To roll back: 1 transactions, 7568214 rows

Friendica Support reshared this.

in reply to Philipp Holzer

Wow, now it's super fast... thanks for the work. Do you have a way to receive donations?

Friendica Support reshared this.


Philipp Holzer reshared this.


#Friendica needs new contributors!

I'm not used to do this, but with the large influx of new users and node admins recently coming from #Twitter, our small team is now behind the curve for handling support requests, bug reports and bring about much-needed features.

The project is built on a #PHP / #MySQL platform, but we also need people to be able to assist others just using the software to give developers some space.

If you're willing and able to help, please follow @Friendica Support and the project on Github: github.com/friendica/friendica

Thank you!

This entry was edited (2 years ago)
Unknown parent

mastodon - Link to source
n8chz

@legeneralmidi En théorie, tous les services fédiversaux sont interopérables, mais mes partages de Mastodon à #Friendica viennent sans texte.

github.com/friendica/friendica…

Friendica Support reshared this.

in reply to n8chz

@n8chz ⒶⒺ Désolé pour ça, ce problème semble être très spécifique à votre situation. Il sera difficile de le résoudre fiablement si nous ne pouvons pas le reproduire fiablement de notre côté.

Friendica Support reshared this.



Questions about /display/


!Friendica Developers

@Michael Vogel - I'm currently moving the mod/display.php

As far as I can see, there are two main-routes for this controller:

  • /display/{guid}
  • /display/feed-item/{uri-id}.atom[/conversation.atom]


and as far as I can say, it would be better if the second route should have an own root, like `dfrn_item/{uri-id}[/conversation.atom`, as it's a DFRN only logic.
Am I right so far?

If so, can I move it to this new path or is this path necessary for the federation usage, means it could be called from other servers too?
I just found calls with prefix DI::baseUrl(), so I guess it's "just" for the server itself, but I'm not sure ...

Friendica Developers reshared this.

in reply to Philipp Holzer

@Michael Vogel / @Hypolite Petovan

I still struggle at mod/update_display.php
It's the only place, where the display_content from mod/display.php is called with the parameter $update = true.

But I don't find any place inside Friendica, where we call /update_display?p=<profile_uid>.

Do you know if this is till in use and if so, what's the meaning of it?
If we wouldn't use it anymore, there's a lot of code at mod/display.php, which I could delete in that case..

Friendica Developers reshared this.



API usage for Friendica Frontend


!Friendica Developers

And another question :-)

Currently, we strictly separate between API calls and the Frontend.
But, Why?

I'm currently moving the events to the Module directory.
And there's already an API for it, but as far as I understand @Michael Vogel , we must not mix API and Frontend.

But as far as I know, in modern environments, there's a backend part (= API), and a frontend part (= view)
The own frontend still uses the same API. So we don't have to implement the same business logic at different places. As I said, I came across at Event, where this issue produces significant double-code.

So at least what I would like (in a final state :-):

  • API ... Create, Update, Delete, Get, GetMultiple for our Data model, or "everything without HTML output"
  • Module ... Everything, which produces HTML output :-)


I know, this would be a BIG change :-)

Friendica Developers reshared this.

in reply to Philipp Holzer

API is a misnomer in our case because it can designate several things:
- Our internal API (what you described in your post)
- The Twitter API endpoints we support
- The Mastodon API endpoints we support
- The GNU Social API endpoints we support
- The Friendica-specific API endpoints we expose (mainly for Friendiqa)

We definitely lack a unified internal API but that’s supposed to be taken care of by the Repository/Factory/Model/Entity structure that we should keep expanding.

This entry was edited (2 years ago)

Friendica Developers reshared this.

in reply to Hypolite Petovan

@Hypolite Petovan
why don't separate the api for Twitter, Mastodon, Gnu etc. and of course an api to communicate with the frontend.
No mixing the api together. Activateing the maybe Mastodon APi as main class with subclasses in one folder and subfolders.

@Philipp Holzer @Michael Vogel

This entry was edited (2 years ago)

Friendica Developers reshared this.

in reply to 👤 𝟳𝗵𝗼𝗺𝗮𝘀 ✅🖖🔒

So far we’ve done a good job at having the business code identical for the Twitter and Mastodon API endpoints. There are some legacy Friendica endpoints (mostly the asynchronous endpoints used by Javascript) that are redundant with the page modules.
in reply to Hypolite Petovan

@Hypolite Petovan
Can this endpoint not be closed and moved to an other area. So that you have the main friendica core who will do the internal magic. On the other hand the API section who is communicating with the core. Additional an internal API who is communicating with the core and the frontend section. On all APIs there is only data communication.

The Frontend Section is an additional area who will communicate with the frontend API and the user.
So the magic on the frontend section will be created only in the frontend and the API communication will only send data ... No html, no Jscript and no other magic.

In that case all three sections (Core, API, Frontend) could be developed separate with less interference between them.

Sure if you need a new function... First core the API and if needed then frontend
@Philipp Holzer @Michael Vogel

Friendica Developers reshared this.

in reply to 👤 𝟳𝗵𝗼𝗺𝗮𝘀 ✅🖖🔒

I don't think we need to go that far. Separating the web frontend from the API would be a massive undertaking at this point, and also there's a trend where even Javascript frontends are starting to request already formed HTML from the backend because it turns out it's faster this way. Which means we're both behind and ahead of the curve!

However we do need to extract the business logic in a unique place so that we can reference it from the places it is duplicated.

Friendica Developers reshared this.

in reply to Hypolite Petovan

@Hypolite Petovan
I think this is very necessary.
Passing preformatted web content makes you very inflexible since you have to go directly to the core. That's where I see the problem, that you either have a good coder for the core but not a good designer. The other way round a designer of the frontend can't write a good core code.
For this reason the Smarty framework is currently running in Friendica, which as I understand the framework wants to separate this.

If you want to speed up the whole thing, you have to get away from PHP code which is executed at runtime and create static pages. Peppered with lots of javascript which offloads/executes the execution to the client.

But ok that's my opinion on the subject.

@Philipp Holzer @Michael Vogel

Friendica Developers reshared this.

in reply to 👤 𝟳𝗵𝗼𝗺𝗮𝘀 ✅🖖🔒

Thankfully, we can have both HTML modules and API endpoints! As long as we don't duplicate business code, which is what Philipp has been encountering.

On the other hand, writing a full Javascript web frontend is a massive endeavor, there currently are almost 500 web frontend routes that each represent a part of a Friendica feature. They all would need to have a Javascript counterpart, without even counting the corresponding bespoke API endpoints that would need to be created.

Friendica Developers reshared this.

Unknown parent

@Michael Vogel
What is the showstopper for that? Cause that is also a thing for a new Webdesign. So The Web sould be separate from the code with all the magic.
@Philipp Holzer

Friendica Developers reshared this.



Add-on refactoring


!Friendica Developers

So this are my first thoughs about the Addons:

  • Metadata about each add-on should be in an extra file, I would prefer something like addon.json
  • I would like to replace the whole Hook class with a proper Registration class, which is loaded when activating the Add-on. So we can remove the whole Hook table.
  • All Events in Friendica should be in an events.config.php with a proper description (maybe as constants?)
  • for legacy purpose, I would create a wrapper class like the LegacyModule class, which would execute the corresponding php-functions
  • I'm thinking about a giving each Event a specific interface (not knowing how tbh currently, maybe the corresponding interface setting as a value for the Event inside the events.config.php?), so each event and each add-on will know exactly what parameters and which result is necessary


This is it so far..

I would like to start with creating the events.config.php and replace all text with constants.. I think this is a non invasive behaviour, but gives some insights about the next steps

Friendica Developers reshared this.

Unknown parent

friendica - Link to source
Hypolite Petovan
The folder structure won’t change, but instead of a list of functions, addons will become classes.

Friendica Developers reshared this.



Friendica codebase restructure


!Friendica Developers

Based on the PR github.com/friendica/friendica…

@Hypolite Petovan is right, I should have started this conversation first, but it made fun to restructure our code, so it is what it is *gg*

What's the issue/my pain:

I'm thinking about introducing the PSR-7 Request/Response interfaces and create an Emitter, so there's no way out anymore directly inside the code, everything is done by the emitter. With this pattern, we could add all the authentication stuff inside a "Middleware" and get rid of the different auth-places (App, Security\Authentication, Module\BaseApi)

So, now I tried to think where to start. We would need a Middleware directory for all the auth/base-load things, but we already do have a mix of domain-based directories (TowFactorAuth), functionallity based directories (Model/Object/Factory/Collection/Capabilities) and and core directories (Core, App, Console, Worker)...

And I try to start to "really" encapsulate the domain-based directories inside Friendica\Library, so we get rid of some mixings, having all domain based classes inside one directory.

In the end, I think we should have some core directories at src, directing everyone to the right place.

I hope I didn't loose you at least, I just wrote down my path to this PR :)

PS: there're "just" ~200 code changes, the rest is because of the messages.po recreation don't worry :)

Friendica Developers reshared this.

in reply to Hypolite Petovan

So do you agree with the restructuring or shall I postpone it after mod/ is restructured? I think first, but I'm not sure :-D

Friendica Developers reshared this.



Friendica Woodpecker / CI in GitHub bricked


!Friendica Developers

I'm sorry, I bricked the CI with the latest Woodpecker change.. Now instead that we don't create Archives at all nodes, we now don't execute some pipelines at all and they are "stuck forever" and blocking new executions.

I'm not sure if I can fix it today, so here's the announcement :-/

I will kick the pipelines from time to time to make PR checks possible, but the "merge" pipelines aren't working (so no new archive appears currently..)

Friendica Developers reshared this.

in reply to Philipp Holzer

I think I got it! :)

It was a mix of wrong environment variable setup for $WOODPECKER_LABELS (added additional " for the value, which got escaped and added to the value) and using the `label` filter in Woodpecker, not knowing that this feature is only in the nightly builds yet

--> as far as I can see, all Archives are now deployed at files.friendi.ca as expected :)

Friendica Developers reshared this.



Yet another Git-Repo :D


!Friendica Developers

@Tobias - I'd like to commit changes for the computed "docs" page at git.friendi.ca/friendica/docs.…

Can you create this repo for me and make me admin for it? Thx :-)

Friendica Developers reshared this.



docs.friendi.ca


!Friendica Developers

Hi, I'd like to add a new domain `docs.friendi.ca` where we store the whole documentation. It should be versioned, like https://docs.friendi.ca/2022.09/ for the docs of the stable 2022.09 version or https://docs.friendi.ca/develop/ for the docs of the develop branch.

I think it would be possible to automatically generate it.

But before, I do need the domain - pinging @Michael Vogel - and a routing to the current files.friendi.ca vm-instance - pinging @utzer ~Friendica~ :-)

I'd like to generate it automatically during the CI, so the update would be smooth and automatically :-)

Friendica Developers reshared this.

Unknown parent

friendica (DFRN) - Link to source
Philipp Holzer
I don't have currently that time, so it's still WIP .. When it's finished, of course there will be a sub-dir for stable releases and develop - and Release candidates, similar to our branch structure :)

Friendica Developers reshared this.



2 Factor & DB plaintext


!Friendica Developers

Hi all, I lost my mobile phone during the last festival and so my TOTP-App was gone forever. Unfortunately, I wasn't aware of my recovery codes, so I thought I would have to reset my accounts of Friendica.

But no! just use select code from 2fa_recorvery_codes where uid = 66 and used is NULL; and voilá, I used the first code and was back in.

@Hypolite Petovan , isn't this a possible security issue, is it?? Storing such sensible data as plaintext. I think we should save it as hash like for passwords to make it impossible to read it again .. Yes, the downside is that there's no possibility to save recovery_codes from the settings-panel again, but tbh I feel a little bit unsafe, but maybe it's just a feeling :-)

Friendica Developers reshared this.

in reply to Philipp Holzer

We hash passwords not just because we want to prevent someone having a database dump from logging in the website depending on this database, but also for the other websites users might use the same password for. There is no such risk for randomly generated recovery codes.

Additionally, the use of the recovery codes necessitates the use of the password (that you still had in your head/password manager), after all it's a second-factor authentication, which means that it can be freely compromised as long as the first factor (the password) is safe.

Still, we can hash these codes and as a result only show them once, it isn't that hard other than requiring someone™ to spend some quality time on this task.

Friendica Developers reshared this.



Large database


!Friendica Admins
Hi everyone,

last week, I struggled with the size of my database for opensocial.at .. It went full (80GB) .. I added additional 10GB to the volume, but it seems to constantly increase and I think in a few weeks, I will be at the same situation like the last week.

Is there a possibility to shrink the database, or to wipe old data?
I recently deleted about 1.200 spam accounts (and set the register option to approval first ... lessons learned...). Are there leftovers inside the db which I can delete?

It's not that I don't have enough space, but I'm afraid of my backup borg-backup space in the near future ^^

Friendica Admins reshared this.

in reply to Philipp Holzer

nope, there're no active relays "Das System hat derzeit keinerlei Relays abonniert."

Friendica Admins reshared this.







in reply to Hypolite Petovan

haha, I think I will try to call her with that name and see what happens :D
Unknown parent

friendica (DFRN) - Link to source
Philipp Holzer

d778af0b0ee3462d6cff943e0a8d8a9a5364c67f
What I did:

  • Clicked at "Friendica Developers" under "Forums" inside the left navbar
  • Opened a new Topic per pencil-button


New Github/Gitea Label


!Friendica Developers
I'd like to introduce `Refactoring` as a new label. I feel like we should distinguish between "real" enhancements and "just" refactorings. What do you say?

Friendica Developers reshared this.

in reply to Philipp Holzer

I feel like most current issue labels are meaningless beyond the Bug label we use to prioritize issues during the RC period.

So it wouldn’t do any harm to add another one, but it won’t help with any existing workflow either.

Unknown parent

friendica - Link to source
Hypolite Petovan
We never had any regression issue during refactoring tasks. 🤥


Locks with hostname


!Friendica Developers
Calling @Hypolite Petovan and @Michael Vogel for support at github.com/friendica/friendica…

I do want to alter the way how locks are working to make parallel hosts/nodes possible. I think the logic itself is pretty straight forward and final.

BUT - how can I upgrade the lock table without using it during the update process itself, it's a gordon knot :-/
I think about using a new method at DBStructure: github.com/friendica/friendica… and execute it before using the lock.

Do you think it works? At least my local nodes are working, but I'm totally unsure about it yet ..

I'd like to finish this PR for 2021.12 as another preparation for making Friendica "cluster ready" :D

This entry was edited (3 years ago)

Friendica Developers reshared this.

in reply to Philipp Holzer

Any idea @Michael Vogel ? :) I'm really stuck here and it's one major piece in the puzzle to make Friendica cluster-ready ;-)
Unknown parent

@Michael Vogel @Hypolite Petovan @Friendica Developers but wouldn't that possibly create a race condition, because a 2nd worker would start another update and wouldn't find the lock anymore so update it again?


Questions Depository


!Friendica Developers
Calling @Hypolite Petovan for help :-)

Why did you introduce a separate "Navigation" directory under "src"?

My guess: this is really a DDD practice using a directory for a use cases and define the different class-types under it (creating boundaries between differently purposed code). But if so, why don't you include the Modules/Notification as well?
And "Navigation/Notification" triggers for me, that the focus of this class is only the notifications at the top of the navbar, am I right?
I'm trying to follow your pattern and refactor the ProfileField / PermissionSet.

Would it be a new "src":
"src/Profile" , where PermissionSet and ProfileField are Entities/ValueObjects ? Or how would you structure these repositories/models?

This entry was edited (3 years ago)

Friendica Developers reshared this.

in reply to Philipp Holzer

Aaahh sounds like the "good old" Model View Controller (MVC) pattern:
- Model --> where the business logic is (=> Entity, Depository, ...)
- View --> the presentation layer (=> Renderer / smarty templates)
- Controller --> The logic to show the right business logic (glue between model & view) (=> Modules)
=> And I like it :-)

I'll give it a try after finishing the WebDav class ;-)


Unknown parent

friendica (DFRN) - Link to source
Philipp Holzer

Yes you can :-)

But Disclaimer: I was the only one who tested it yet, so please test it for your own before using it in production



Endlich wieder mal ein cooles Konzert gehört.. 1,5 Jahre ist es her :) - "Last Decission,


A test Post from Friendica


Hi all, I'm trying to reach Twitter with my new Friendica Twitter plugin :-)
in reply to Philipp Holzer

@Hypolite Petovan I got a lot of posts mirrored to my feed as well .. Do you know any other test for twitter plugin?




Mozilla, die bekannt für den Webbrowser Firefox und das Emailprogramm Thunderbird sind, stellen jetzt auch einen öffentlichen Riot ( https://about.riot.im/ ) Matrix Chat Server zur Verfügung

https://chat.mozilla.org/




Friendica Logo for andOTP


With release 0.7.0 (currently beta) and above, the official Friendica Logo is now part of the thumbnail list :)
Android (x) Client (x) 2fa (x) otp (x) andOTP (x)
in reply to Philipp Holzer

Maybe you shouldadd, that andOTP is an Open source two-factor authentication for Android application github.com/andOTP/andOTP




Hey FEDIVERSE 🌏

2 people are currently working on an #opensource federated #reddit alternative, called #lemmy.
They need some open source #developers to help them enable the federating function.. #activitypub

BOOST to help development...

dev.lemmy.ml/post/29164