Slow /photo/preview performance
Hi all,
Because of the ongoing performance issues, I moved my whole Friendica database of opensocial.at and friendica.me to a dedicated root-server just for mariadb with 128GB RAM.
At first, it seemed like it helped and opensocial.at is faster than before (the performance of friendica.me is still a mess, but I will analyze it later), but the /photo/preview route is now going crazy:
It seems like the RAM I/O issue I had (which is now gone) is now replaced by an Network throughput issue - but at least the network I/O seems fine:
Do you have any idea what I can do now?
One thing would be to add a "StorageClass" for the Photo to load them (directly) from a filedirectory/block storage instead of the database.
CI currenrly not usable
Hi, I recently tried to increase the woodpecker / ci performance. Unfortunately it dropped the repositories after migrating from sqlite to MySQL as backend. @Tobias re-added it again (thanks 
) but now some settings are still missing.
So currently, the CI isn't working 
I'll be back home at Sunday evening, but I think I will fix it not before Monday/Tuesday..
So if you need to merge new PRs, keep in mind that the files under files.friendi.ca aren't updating ...
Sorry and I'll give an update when it's working again..
like this
Geplante Downtime / Server Migration
!Opensocial.at
-- English below
Liebe opensocial.at BenutzerInnen,
Ich starte nun eine Migration von opensocial.at auf eine stärkere server-hardware. Dadurch entstehen temporäre Downtimes in den kommenden Stunden.
Ich möchte mich für die Unannehmlichkeiten entschuldigen, aber ich denke, dass dadurch die gesamte Performance von opensocial.at wieder besser werden sollte :).
Lg,
Philipp
----
Dear opensocial.at user,
I'm planning to migrate my server infrastructure onto a more powerful environment.
The goal is to increase the performance, as, I think, you all currently experience a downgrading performance.
I hope it will help :)
It will take some hours, where opensocial.at will be temporary unavailable from time to time.
Regards,
Philipp
Leider hat die Migration nicht geklappt und ich musste wieder zurückrollen.
--> Ich probiere es im August noch einmal
---
Unfortunately, the migration didn't work so I had to rollback.
--> I will retry again in august
Norz Tekh 🏳️🌈 likes this.
Planned Downtime / Migration
Dear friendica.me user,
I'm planning to migrate my server infrastructure onto a more powerful environment.
The goal is to increase the performance, as, I think, you all currently experience a downgrading performance.
I hope it will help :)
It will take some hours, where friendica.me will be temporary unavailable from time to time.
Regards,
Philipp
--> I will retry again in august
- ich habe deinen Account bei opensocial.at abgelehnt, weil er am selben Server läuft wie Friendica.me , mit dem du bereits schon nicht glücklich wurdest.. probier libranet.deDer Motzmichel :mastodon: likes this.
#FediBlock*.mastotroll.netz.org♲ Admin (of friendica.utzer.de) - 2023-05-03 18:16:56 GMT
New troll domain with many subsomains. This is a strongly recommended block for all #Fediverse services for *.mastotroll.netz.org.Please share this post!
Torsten Torsten likes this.
Slow photo requests / inbox requests
Hi !Friendica Admins ,
I'm currently tracing down performance issues, I' currently suffering from on my nodes opensocial.at and friendica.me .
It seems like the main performance problems are
- delivering photos
- [code]/inbox/code] requests
You can see some peeks and some long runners here:
The long runners (10sec) are the /inbox requests, the peaks are delivering photos after opening conversations, network pane, ...
Do you have any hints how I could improve these two types of requests?
The database is currently using ~50GB RAM, I set them as high as possible.
'avatar_cache' => true,
'avatar_cache_path' => '/path/to/a/folder',But that one is tricky to set up. Best is to store the photos in a folder outside the web folder and then you can define a rule in your webserver that will redirect requests to the avatar folder to that folder.
Roland Häder likes this.
Roland Häder likes this.
Philipp Holzer likes this.
/photo/contact route for the avatar cache, don't I?
No, the setup process is completely different. You have to have a path (somewhere) that is both readable and writable by the frontend and backend process
This path needs to be reachable when you call https://your.server.tld/avatar (you have to redirect /avatar) or you have to define avatar_cache_url.
You you setup the file path from above in avatar_cache_path and set avatar_cache to true. After you deactivate the caching of the avatars in the admin frontend, it should work.
The URL path is stored in the contact table fields. So once you defined that URL, you mustn't change it again, since all old requests would fail. This mechanism is powerful, but currently too easy to misconfigure, that's why it is only accessible via the config file.
friendica/Avatar.php at develop · friendica/friendica
Friendica Communications Platform. Contribute to friendica/friendica development by creating an account on GitHub.GitHub
like this
https://? That's never a good idea.
seems to work : https://friendica.philipp.info/avatar/4c/a1/d7c/53/8b/f08b/cd3c2904ac0bfd56-300.png?ts=1682970014
This image is now served by a completely different nginx than the frontend nginx . Because it's stateless, I'm now able to start more nginx for "just" serving stateless static content!
Brilliant!
Philipp Holzer likes this.
17000 from 59000 finished ;)Michael Vogel likes this.
avatar/, there are still a lot of long running requests, see the image above. Do you have any idea? The overall CPU/RAM is on a normal state.
Friendica 2023.04 released
Content warning: We are very happy to announce the availability of the new stable release of Friendica “Giant Rhubarb” 2023.04. The highlights of this release are For details, please the CHANGELOG file in the repository. What is Friendica Friendica is a decentralized comm
like this
reshared this
rel="me" attribute and points to your user profile on Friendica. Then add a link to your homepage in your profile. https://wiki.friendi.ca/docs/verify_homepage
Update JSON LD
@Hypolite Petovan already fixed a current JSON-LD issue with https://git.friendi.ca/friendica/php-json-ld/pulls/1
Do we need a 1.1.2 release to upgrade it in the Friendica composer.lock file as well?
The logs of my instances are flooded with this error, that's why I'm asking *g*
Add support for local files for unsecured document loading
- $http_response_header isn't set in the local file read case Address https://github.com/friendica/friendica/issues/12728#issuecomment-1407623586Friendica
like this
BaseURL components (hostname, SSL policy, urlpath)
I'm currently reducing the whole
BaseUrl.php code massively.Do we really need the ssl_policy, urlpath and hostname separate from the
system.url? I will use for the
BaseUrl.php, based on the system.url a "real" UriInterface as $this->url, so we don't need saving the scheme, urlpath and hostname separately anymore. They are just useful for the install process but must not be changed afterwards.The only thing, which I'm unsure is the
ssl_policy, because if someone changes it afterwards in the admin site, all URL in all contacts and photos will get updated. But the question is => is this even allowed? I think this could brick the access over federation because the base-url of each entry isn't right anymore. And it isn't supported when the policy is changed by console.So I would drop it as well and merge all config entries into the
system.url.Additionally, I will replace the Exception with a "CRITICAL" log entry to avoid a WSOD.
https://friendica.local you definitely need to ignore SSL/certificate errors because they are mostly only self-signed.
https and there is no check at all in the code that would fail here. It works fine Roland Häder likes this.
Friendica.me | Opensocial.at Short Downtime because of DB adjustements
I do have to restart the Database instances because they are too resource hungry.
There's a short downtime possible.
reshared this
Friendica 2023.01 released
Content warning: Christian Pöschl from usd AG has found another XSS vulnerability in Friendica which is close with this hotfix release of Friendica. In addition some other bugfixes for the distribution of forum postings and improvements to the update process of node infor
like this
reshared this
Friendica 2022.12 released
Content warning: We are very happy to announce the avail-ability of the new stable version of Friendica. Wrapping up the sprint from the 2022.10 release of Friendica we closed 73 filed issues and had almost 300 pull requests by 19 contributors. A special thanks goes out t
like this
reshared this
.htaccess file needed to be updated on Apache. If I'm upgrading from 2022.10, do I need to do this still?
@Hypolite Petovan
.htaccess file, yes.Jonathan Lamothe likes this.
opensocial.at database data seems corrupt
It seems like I do have a problem with my opensocial.at database
.The database ran full about a year ago (what a shame, I know :( ...) and I had to recover some of the data (otherwise I had to import a backup about ~12hours ago, I tried to avoid to loose posts for this period of time for my users ...). So I stopped it, started it with
innodb_force_recovery=3, repaired it and checked it with mysqlcheck.Two days ago, I updated the mariadb from 1.10.3 to 1.10.10 and now the problems occur again.
I stopped the MariaDB 1.10.3 container, updated it to 1.10.10 and started it again .. And MariaDB started a "crash recovery". I really don't know why... The crash recovery wasn't successfully (I tried it ~4 times), so I had to add
innodb_force_recovery=3 again.Now the database went up, but everytime I stopped it and started it again, the crash recovery appears again.
So after the instance was up and running, I dumped the whole database with
mysqldump into one single *.sql , started a brand new MariaDB 1.10.10 and imported the dump again.So far so good ..
But... unfortunately, after a restart, the crash recovery appears again. So I'm totally lost, what's now happening..
I noticed during the import that one batch of rows took about 2 hours to complete with the result query affected 0 rows.
here's my
customized.cnf, which I'm using (it's a Hetzner root server with 64 GB RAM and 12 CPUs):[mysqld]
query_cache_size = 0
query_cache_type = 0
performance_schema = ON
join_buffer_size = 140M
innodb_buffer_pool_size = 12G
innodb_log_buffer_size = 31M
innodb_log_file_size = 3G
table_open_cache = 1000
max_connections = 400
wait_timeout = 200
interactive_timeout = 4000
log-bin = mysqld-bin
transaction-isolation = READ-COMMITTED
binlog-format = ROW
skip-innodb-read-only-compressed = ON
innodb_read_only_compressed = OFF
innodb_use_native_aio = OFF
innodb_fast_shutdown=0
innodb_max_dirty_pages_pct=0
innodb_buffer_pool_dump_at_shutdown=1
innodb_buffer_pool_load_at_startup=1For the new MariaDB instance, I didn't add any customization to avoid any wrong options.
currently, it makes a recovery again:
db_1 | 2022-12-16 10:41:00 0 [Note] InnoDB: Rolled back recovered transaction 371172
db_1 | 2022-12-16 10:41:11 0 [Note] InnoDB: To roll back: 1 transactions, 7611737 rows
db_1 | 2022-12-16 10:41:26 0 [Note] InnoDB: To roll back: 1 transactions, 7608132 rows
db_1 | 2022-12-16 10:41:41 0 [Note] InnoDB: To roll back: 1 transactions, 7603940 rows
db_1 | 2022-12-16 10:41:46 0 [Note] DDL_LOG: Crash recovery executed 1 entries
db_1 | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '0.0.0.0'.
db_1 | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '::'.
db_1 | 2022-12-16 10:41:47 0 [Note] mariadbd: ready for connections.
db_1 | Version: '10.10.2-MariaDB-1:10.10.2+maria~ubu2204' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
db_1 | 2022-12-16 10:41:56 0 [Note] InnoDB: To roll back: 1 transactions, 7601453 rows
db_1 | 2022-12-16 10:42:11 0 [Note] InnoDB: To roll back: 1 transactions, 7600146 rows
db_1 | 2022-12-16 10:42:26 0 [Note] InnoDB: To roll back: 1 transactions, 7598395 rows
db_1 | 2022-12-16 10:42:41 0 [Note] InnoDB: To roll back: 1 transactions, 7595967 rows
db_1 | 2022-12-16 10:42:56 0 [Note] InnoDB: To roll back: 1 transactions, 7594000 rows
db_1 | 2022-12-16 10:43:11 0 [Note] InnoDB: To roll back: 1 transactions, 7590646 rows
db_1 | 2022-12-16 10:43:26 0 [Note] InnoDB: To roll back: 1 transactions, 7585618 rows
db_1 | 2022-12-16 10:43:41 0 [Note] InnoDB: To roll back: 1 transactions, 7579929 rows
db_1 | 2022-12-16 10:43:56 0 [Note] InnoDB: To roll back: 1 transactions, 7574058 rows
db_1 | 2022-12-16 10:44:11 0 [Note] InnoDB: To roll back: 1 transactions, 7568214 rowslike this
xy..
in reply to Philipp Holzer • • •Fatal Error (E_ERROR): gd-webp cannot allocate temporary bufferall the time.. could this be related to the "/photo/preview route" ?Philipp Holzer
in reply to xy.. • • •xy..
in reply to Philipp Holzer • • •/var/www/localhost/friendica/src/Object/Image.phpline 171Michael Vogel
in reply to Philipp Holzer • • •Philipp Holzer
in reply to Michael Vogel • • •Roland Häder
in reply to Philipp Holzer • • •/tmpfolder? Here on my server it ran full withmagick-XXXXXfiles.Philipp Holzer
in reply to Roland Häder • • •But there are just two zero-byte magick files at
/tmp:)