Skip to main content



#FediBlock *.mastotroll.netz.org

♲ Admin (of friendica.utzer.de) - 2023-05-03 18:16:56 GMT

New troll domain with many subsomains. This is a strongly recommended block for all #Fediverse services for *.mastotroll.netz.org.Please share this post!

#Friendica @Friendica Admins
cc @utzer [Friendica]




Friendica Admins reshared this.


Slow photo requests / inbox requests


Hi !Friendica Admins ,

I'm currently tracing down performance issues, I' currently suffering from on my nodes opensocial.at and friendica.me .

It seems like the main performance problems are

  • delivering photos
  • [code]/inbox/code] requests


You can see some peeks and some long runners here:

The long runners (10sec) are the /inbox requests, the peaks are delivering photos after opening conversations, network pane, ...

Do you have any hints how I could improve these two types of requests?

The database is currently using ~50GB RAM, I set them as high as possible.

Friendica Admins reshared this.

For the photos you can define the avatar cache:
'avatar_cache' => true,
 'avatar_cache_path' => '/path/to/a/folder',

But that one is tricky to set up. Best is to store the photos in a folder outside the web folder and then you can define a rule in your webserver that will redirect requests to the avatar folder to that folder.

Friendica Admins reshared this.

@Michael Vogel @Philipp Holzer Will these cached avatars be checked if they are outdated? I currently have trouble getting @Tobias 's avatar over here.

Friendica Admins reshared this.

@Michael Vogel @Philipp Holzer why is it necessary to have the folder outside of the web folder?

Friendica Admins reshared this.

It will become extremely large over time. So storing in some separate folder might be better for backup purposes or for keeping your web root clean. Also there is an option for an alternate URL. So you could even store these files on some separate web server that only serves static content.

Friendica Admins reshared this.

@Michael Vogel @Philipp Holzer is the avatar cache necessary for the workers too or just for the frontend app?

Friendica Admins reshared this.

These two don't depend on each other.

Friendica Admins reshared this.

So I do need to redirect only the /photo/contact route for the avatar cache, don't I?

Friendica Admins reshared this.

No, the setup process is completely different. You have to have a path (somewhere) that is both readable and writable by the frontend and backend process
This path needs to be reachable when you call https://your.server.tld/avatar (you have to redirect /avatar) or you have to define avatar_cache_url.

You you setup the file path from above in avatar_cache_path and set avatar_cache to true. After you deactivate the caching of the avatars in the admin frontend, it should work.

The URL path is stored in the contact table fields. So once you defined that URL, you mustn't change it again, since all old requests would fail. This mechanism is powerful, but currently too easy to misconfigure, that's why it is only accessible via the config file.

Friendica Admins reshared this.

I think I got it. I try it with my friendica.philipp.info node first and check it :-)

Friendica Admins reshared this.

@Michael Vogel @Philipp Holzer U oh, writable and readable by the frontend, means web server? Do you mean here also through https://? That's never a good idea.

Friendica Admins reshared this.

seems to work : https://friendica.philipp.info/avatar/4c/a1/d7c/53/8b/f08b/cd3c2904ac0bfd56-300.png?ts=1682970014

This image is now served by a completely different nginx than the frontend nginx . Because it's stateless, I'm now able to start more nginx for "just" serving stateless static content!

Brilliant!

Friendica Admins reshared this.

Old avatars still will be served the old way and will be replaced, once they are updated - which can take weeks. But I guess I wrote some script that speeds up this process.

Friendica Admins reshared this.

currently, "movetoavatarcache" is running on my node :-) 17000 from 59000 finished ;)

Friendica Admins reshared this.

Hi, after moving all contact images to avatar/, there are still a lot of long running requests, see the image above. Do you have any idea? The overall CPU/RAM is on a normal state.

Philipp Holzer reshared this.


Friendica 2023.04 released


Content warning: We are very happy to announce the availability of the new stable release of Friendica “Giant Rhubarb” 2023.04. The highlights of this release are For details, please the CHANGELOG file in the repository. What is Friendica Friendica is a decentralized comm

hey.......i would like to know how to get verified on #Friendica .... if anyone would tell me, that would be cool....
@Expert Plus 🍥 put a link on your homepage that has a rel="me" attribute and points to your user profile on Friendica. Then add a link to your homepage in your profile. https://wiki.friendi.ca/docs/verify_homepage
But I don't have a website. What about using GitHub?
@Expert Plus 🍀 if that adds a rel=me to the link location, than that link should be all that is needed to get a verification.

Friendica Developers reshared this.


Update JSON LD


!Friendica Developers

@Hypolite Petovan already fixed a current JSON-LD issue with https://git.friendi.ca/friendica/php-json-ld/pulls/1

Do we need a 1.1.2 release to upgrade it in the Friendica composer.lock file as well?

The logs of my instances are flooded with this error, that's why I'm asking *g*

Friendica Developers reshared this.

I create a release and a corresponding PR at Friendica itself :)

Friendica Developers reshared this.


BaseURL components (hostname, SSL policy, urlpath)


!Friendica Developers

I'm currently reducing the whole BaseUrl.php code massively.
Do we really need the ssl_policy, urlpath and hostname separate from the system.url?

I will use for the BaseUrl.php, based on the system.url a "real" UriInterface as $this->url, so we don't need saving the scheme, urlpath and hostname separately anymore. They are just useful for the install process but must not be changed afterwards.

The only thing, which I'm unsure is the ssl_policy, because if someone changes it afterwards in the admin site, all URL in all contacts and photos will get updated. But the question is => is this even allowed? I think this could brick the access over federation because the base-url of each entry isn't right anymore. And it isn't supported when the policy is changed by console.

So I would drop it as well and merge all config entries into the system.url.

Additionally, I will replace the Exception with a "CRITICAL" log entry to avoid a WSOD.

Friendica Developers reshared this.

@Philipp Holzer For a local installation, like https://friendica.local you definitely need to ignore SSL/certificate errors because they are mostly only self-signed.

Friendica Developers reshared this.

I know and I'm using local Vagrant with https and there is no check at all in the code that would fail here. It works fine :-)

Opensocial.at reshared this.


Friendica.me | Opensocial.at Short Downtime because of DB adjustements


!friendica.me , !Opensocial.at

I do have to restart the Database instances because they are too resource hungry.

There's a short downtime possible.

friendica.me reshared this.

friendica.me reshared this.

Restart finished

Opensocial.at reshared this.


Philipp Holzer reshared this.


Friendica 2023.01 released


Content warning: Christian Pöschl from usd AG has found another XSS vulnerability in Friendica which is close with this hotfix release of Friendica. In addition some other bugfixes for the distribution of forum postings and improvements to the update process of node infor

There is one typo in the text that mentions 2022.12.
I found it too, but didn't want to fix an "official typo" :D

Philipp Holzer reshared this.


Friendica 2022.12 released


Content warning: We are very happy to announce the avail-ability of the new stable version of Friendica. Wrapping up the sprint from the 2022.10 release of Friendica we closed 73 filed issues and had almost 300 pull requests by 19 contributors. A special thanks goes out t

news@forum.friendi.ca I remember the 2022.12-rc version saying that the .htaccess file needed to be updated on Apache. If I'm upgrading from 2022.10, do I need to do this still?
@Jonathan "Mastodon" Lamothe Yes, very much so. It only affects one specific API call, but it would be difficult to troubleshoot.
@Jonathan "Mastodon" Lamothe just pull/update and then quickly look in the shipped .htaccess-dist or so and there is a rewrite rule line with an additional B compared to your current version, so only one capital letter B needs to be added.

@Hypolite Petovan
@Jonathan "Mastodon" Lamothe @utzer ~Friendica~ If you didn't add any changes to your local .htaccess file, yes.
@Hypolite Petovan Out of curiosity, is there a reason I would have manually changed it?

Friendica Support reshared this.


opensocial.at database data seems corrupt


!Friendica Support

It seems like I do have a problem with my opensocial.at database :-( .

The database ran full about a year ago (what a shame, I know :( ...) and I had to recover some of the data (otherwise I had to import a backup about ~12hours ago, I tried to avoid to loose posts for this period of time for my users ...). So I stopped it, started it with innodb_force_recovery=3, repaired it and checked it with mysqlcheck.

Two days ago, I updated the mariadb from 1.10.3 to 1.10.10 and now the problems occur again.

I stopped the MariaDB 1.10.3 container, updated it to 1.10.10 and started it again .. And MariaDB started a "crash recovery". I really don't know why... The crash recovery wasn't successfully (I tried it ~4 times), so I had to add innodb_force_recovery=3 again.

Now the database went up, but everytime I stopped it and started it again, the crash recovery appears again.

So after the instance was up and running, I dumped the whole database with mysqldump into one single *.sql , started a brand new MariaDB 1.10.10 and imported the dump again.

So far so good ..

But... unfortunately, after a restart, the crash recovery appears again. So I'm totally lost, what's now happening..

I noticed during the import that one batch of rows took about 2 hours to complete with the result query affected 0 rows.

here's my customized.cnf, which I'm using (it's a Hetzner root server with 64 GB RAM and 12 CPUs):
[mysqld]
query_cache_size = 0
query_cache_type = 0
performance_schema = ON
join_buffer_size = 140M
innodb_buffer_pool_size = 12G
innodb_log_buffer_size = 31M
innodb_log_file_size = 3G
table_open_cache = 1000
max_connections = 400
wait_timeout = 200
interactive_timeout = 4000

log-bin = mysqld-bin
transaction-isolation = READ-COMMITTED
binlog-format = ROW
skip-innodb-read-only-compressed = ON
innodb_read_only_compressed = OFF
innodb_use_native_aio = OFF

innodb_fast_shutdown=0
innodb_max_dirty_pages_pct=0
innodb_buffer_pool_dump_at_shutdown=1
innodb_buffer_pool_load_at_startup=1


For the new MariaDB instance, I didn't add any customization to avoid any wrong options.


currently, it makes a recovery again:
db_1     | 2022-12-16 10:41:00 0 [Note] InnoDB: Rolled back recovered transaction 371172                                                                                                                                                                                                    
db_1     | 2022-12-16 10:41:11 0 [Note] InnoDB: To roll back: 1 transactions, 7611737 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:26 0 [Note] InnoDB: To roll back: 1 transactions, 7608132 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:41 0 [Note] InnoDB: To roll back: 1 transactions, 7603940 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:46 0 [Note] DDL_LOG: Crash recovery executed 1 entries                                                                                                                                                                                                          
db_1     | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '0.0.0.0'.                                                                                                                                                                                                             
db_1     | 2022-12-16 10:41:46 0 [Note] Server socket created on IP: '::'.                                                                                                                                                                                                                  
db_1     | 2022-12-16 10:41:47 0 [Note] mariadbd: ready for connections.                                                                                                                                                                                                                    
db_1     | Version: '10.10.2-MariaDB-1:10.10.2+maria~ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution                                                                                                                                               
db_1     | 2022-12-16 10:41:56 0 [Note] InnoDB: To roll back: 1 transactions, 7601453 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:11 0 [Note] InnoDB: To roll back: 1 transactions, 7600146 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:26 0 [Note] InnoDB: To roll back: 1 transactions, 7598395 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:41 0 [Note] InnoDB: To roll back: 1 transactions, 7595967 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:42:56 0 [Note] InnoDB: To roll back: 1 transactions, 7594000 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:11 0 [Note] InnoDB: To roll back: 1 transactions, 7590646 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:26 0 [Note] InnoDB: To roll back: 1 transactions, 7585618 rows                                                                                                                                                                                                  
db_1     | 2022-12-16 10:43:41 0 [Note] InnoDB: To roll back: 1 transactions, 7579929 rows
db_1     | 2022-12-16 10:43:56 0 [Note] InnoDB: To roll back: 1 transactions, 7574058 rows
db_1     | 2022-12-16 10:44:11 0 [Note] InnoDB: To roll back: 1 transactions, 7568214 rows

Friendica Support reshared this.

Wow, now it's super fast... thanks for the work. Do you have a way to receive donations?

Friendica Support reshared this.

@Torsten Torsten
Me?
No donations needed. I’m just happy to help :)

Philipp Holzer reshared this.


#Friendica needs new contributors!

I'm not used to do this, but with the large influx of new users and node admins recently coming from #Twitter, our small team is now behind the curve for handling support requests, bug reports and bring about much-needed features.

The project is built on a #PHP / #MySQL platform, but we also need people to be able to assist others just using the software to give developers some space.

If you're willing and able to help, please follow @Friendica Support and the project on Github: https://github.com/friendica/friendica

Thank you!
This entry was edited (6 months ago)