| View previous topic :: View next topic |
| Author |
Message |
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
 Posted: Tue Jul 23, 2013 6:52 pm Post subject: TIP: Complete network stack without net.* scripts |
 |
|
Hi all.
It is possible to have a working network stack without resorting to Gentoo net.* scripts at all on simple configuration. Let's agree on what «simple» is:- only hardware interfaces are needed, i.e. no virtual interfaces;
- IP addresses delivered through DHCP (eventually APIPA)
In such conditions all hardware interfaces can be configured, even the loopback interface, lo.
Basic configuration
Install Roy Marples' dhcpcd with zeroconf support:
| Code: | | USE=zeroconf emerge -a dhcpcd |
It is of course best to add the USE flag to your favourite /etc/portage/package.use file. Now append the following lines to dhcpcd configuration file:
| /etc/dhcpcd.conf: | # In some cases the loopback interface won't be up, this directive fixes it:
allowinterfaces lo *
# Configure loopback adapter here
interface lo
static ip_address=127.0.0.1/8 |
Remove all network scripts from all runlevels:
| Code: | rc-update del net.lo boot
rc-update del net.eth0 boot |
Add dhcpcd to the boot runlevel:
| Code: | | rc-update add dhcpcd boot |
Wireless interfaces
If you installed wpa_supplicant, just add it to the boot runlevel, dhcpcd will detect the interface, whatever it is called and set its IP address automatically. Also note a nice thing Linux does when having wired and wireless interfaces together on the same network is that the one with the lowest metric takes precedence over existing connections.
Suppose for instance you have an ongoing transfer of a very large file over your wireless interface to an existing network mountpoint. Plug the cable and wait for dhcpcd to set the IP address of the ethernet interface and you'll see the network activity switch to the fastest interface. You don't even need to interrupt and restart the transfer. That feature comes handy when you cannot bond interfaces, especially with wireless.
VPN interfaces
Now dhcpcd will attach to any visible network interface and assign an IP address whenever it can as soon as it discovers one (it dynamically detects them). If you have VPN interfaces dhcpcd will also interfere, which is probably not what you want. You can either constraint dhcpcd to a list of interfaces:
| /etc/dhcpcd.conf: | | allowinterfaces eth* wlan0 |
or exclude a list of interfaces you want it to not touch at all:
| /etc/dhcpcd.conf: | | denyinterfaces ppp0 en* |
See dhcpcd.conf man pages for more info. If no DHCP server can be contacted for a specific interface an IPv4LL will be assigned. I personally prefer dhcpcd to bind to all interfaces it finds but to the ones I exclude. YMMV.
DNS resolution
Your VPN might be configured with the USEPEERDNS parameter, which affects how DNS resolution works. In general it's a good idea to leave it as it is since being connected to the internet *and* to a private network creates a security hole if traffic is also routed outside the VPN. However if you know what you're doing and still want DNS resolution while using your remote network, here's what to do.
Install dnsmasq and openresolv and create a directory to store name service resolvers for dnsmasq, openresolv will update them as soon as VPN interfaces are created using the remote DNS IP addresses. For instance I stored those files in /etc/dnsmasq.d/ .
| Code: | # mkdir /etc/dnsmasq.d
# rc-update add dnsmasq boot |
Now update resolvconf.conf and dnsmasq configuration.
| /etc/resolvconf.conf: | # Configuration for resolvconf(8)
# See resolvconf.conf(5) for details
resolv_conf=/etc/resolv.conf
dnsmasq_conf=/etc/dnsmasq.d/openresolv.conf
dnsmasq_resolv=/etc/dnsmasq.d/resolv.conf
# If you run a local name server, you should uncomment the below line and
# configure your subscribers configuration files below.
name_servers=127.0.0.1 |
| /etc/dnsmasq.conf: | resolv-file=/etc/dnsmasq.d/resolv.conf
conf-file=/etc/dnsmasq.d/openresolv.conf |
dnsmasq won't start yet if it doesn't find openresolv files in /etc/dnsmasq.d.
| Code: | touch /etc/dnsmasq.d/openresolv.conf
touch /etc/dnsmasq.d/resolv.conf |
dfnsmasq configuration file is huge. You can check it for what it does. See also the man pages for more information and examples.
Update, October 2014: The loopback interface is no longer up by default and needs to be somehow. By default dhcpcd doesn't bring the loopback interface up unless instructed to, which is why dhcpcd.conf now includes this line: | Code: | | allowinterfaces lo * |
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Last edited by VinzC on Tue Nov 18, 2014 10:08 am; edited 9 times in total |
|
| Back to top |
|
 |
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 547
Location: NRW, Germany
|
| Posted: Tue Jul 23, 2013 9:16 pm Post subject: Re: TIP: Complete network stack without net.* scripts |
|
|
| VinzC wrote: | | /etc/dhcpcd.conf: | # Configure loopback adapter here
interface lo
static ip_address=127.0.0.1/8 |
|
Are you sure this is required? |
|
| Back to top |
|
|
dmpogo
Advocate
Joined: 02 Sep 2004
Posts: 3440
Location: Canada
|
| Posted: Tue Jul 23, 2013 10:03 pm Post subject: |
|
|
| Does it detect interfaces if they become available later (like ifplugd) or after radios were off and then on again ? |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Tue Jul 23, 2013 11:23 pm Post subject: Re: TIP: Complete network stack without net.* scripts |
|
|
| /etc/dhcpcd.conf: | # Configure loopback adapter here
interface lo
static ip_address=127.0.0.1/8 |
| Dr.Willy wrote: | | Are you sure this is required? |
Never too sure. I assumed since Gentoo net scripts do assign an IP address to the loopback adapter then it is needed somehow. I didn't check without this said.
| dmpogo wrote: | | Does it detect interfaces if they become available later (like ifplugd) or after radios were off and then on again ? |
Yes, it does.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Wed Jul 24, 2013 2:29 am Post subject: |
|
|
| Vinz: do you have/can you think of, a similar setup for when we're using a static IP, or is net.eth0 always required in order to tell it the IP/mask and gateway? |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Wed Jul 24, 2013 9:59 am Post subject: |
|
|
| steveL wrote: | | Vinz: do you have/can you think of, a similar setup for when we're using a static IP, or is net.eth0 always required in order to tell it the IP/mask and gateway? |
You can check dhcpcd man pages and the static keyword for that, just like I did with lo:
| Code: | interface eth0
static ip_address=192.168.0.10/24
static routers=192.168.0.1
static domain_name_servers=192.168.0.1 |
I've been surprised how rich dhcpcd and its manpages are as it can virtually do, well, almost anything (just wondered about coffee  ).
| man dhcpcd.conf: | static value
Configures a static value. If you set ip_address then dhcpcd
will not attempt to obtain a lease and just use the value for the
address with an infinite lease time.
Here is an example which configures a static address, routes and
dns.
interface eth0
static ip_address=192.168.0.10/24
static routers=192.168.0.1
static domain_name_servers=192.168.0.1
Here is an example for PPP which gives the destination a default
route. It uses the special destination keyword to insert the
destination address into the value.
interface ppp0
static ip_address=
destination routers |
If you don't use static [and no DHCP server exists in the network] dhcpcd will fall back on automatic private local addresses like 169.254.x.y (link-local IP or IPv4LL). And from my observations that kind of addressing seems consistent across reboots. The latter needs further checking though.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Last edited by VinzC on Sat Aug 10, 2013 8:15 am; edited 1 time in total |
|
| Back to top |
|
|
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 547
Location: NRW, Germany
|
| Posted: Wed Jul 24, 2013 11:25 am Post subject: Re: TIP: Complete network stack without net.* scripts |
|
|
| VinzC wrote: | | /etc/dhcpcd.conf: | # Configure loopback adapter here
interface lo
static ip_address=127.0.0.1/8 |
| Dr.Willy wrote: | | Are you sure this is required? |
Never too sure. I assumed since Gentoo net scripts do assign an IP address to the loopback adapter then it is needed somehow. I didn't check without this said. |
I was asking, because I was runnig a dhcpcd-only setup for quite a while and basically all I did was remove all gentoo netscripts and start dhcpcd.
I was wondering if I was missing something or this only worked due to my specific setup. |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Wed Jul 24, 2013 12:37 pm Post subject: Re: TIP: Complete network stack without net.* scripts |
|
|
| Dr.Willy wrote: | I was asking, because I was runnig a dhcpcd-only setup for quite a while and basically all I did was remove all gentoo netscripts and start dhcpcd.
I was wondering if I was missing something or this only worked due to my specific setup. |
What I am pretty sure of is that even if it's useless its purpose then falls back to being purely documentary so cannot harm at all.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
djdunn
l33t
Joined: 26 Dec 2004
Posts: 812
|
| Posted: Sun Jul 28, 2013 6:57 pm Post subject: |
|
|
awesome thanks for this info, im gonna try it out here as soon as this kernel is done compiling
_________________
“Music is a moral law. It gives a soul to the Universe, wings to the mind, flight to the imagination, a charm to sadness, gaiety and life to everything. It is the essence of order, and leads to all that is good and just and beautiful.”
― Plato |
|
| Back to top |
|
|
shad0w_GR
n00b
Joined: 06 Apr 2011
Posts: 16
|
| Posted: Sun Jul 28, 2013 7:07 pm Post subject: |
|
|
| What about pre/post up/down functions? I don't see whether this way is simpler to openrc. |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Fri Aug 09, 2013 8:38 am Post subject: |
|
|
| shad0w_GR wrote: | | What about pre/post up/down functions? |
Fact is dhcpcd has hooks — never tried hence can't tell how it works. Not yet. It might be what you're seeking.
| shad0w_GR wrote: | | I don't see whether this way is simpler to openrc. |
Well, no conf.d/net, no init.d/net.*, all interfaces are configured as soon as they appear without doing anything... You're free to stick to OpenRC if it meets your requirements.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
smartass
Apprentice
Joined: 04 Jul 2011
Posts: 189
Location: right behind you ... (you did turn around, didn't you?)
|
| Posted: Fri Aug 09, 2013 10:40 am Post subject: |
|
|
Thank you for sharing this howto VinzC 
I'm using OpenRC with dhcpcd and ifplugd and noticed that it's almost redundant, because dhcpcd can do what ifplugd does and having several dhcpcd processes started by OpenRC means you get more default gateways and metrics may not be optimal.
However, a major issue for me with such an automagic dhcpcd setup is that I'm not sure how easy it would be to remove an interface from dhcpcd managed pool and configure it temporarily for some debugging, e.g. some static IP or MAC address.
Are you aware of a way to do this with this automagic dhcpcd exclusive setup?
The only way I could think of is making changes to dhcpcd.conf and then sending some signal to the dhcpcd process to re-read the config file.
However, I couldn't find any signal interface in man dhcpcd, but --rebind seems close, but it will temporarily disable ALL interfaces, I was hoping for something that would operate only on a specific interface and wouldn't interrupt others. --release would operate on the specified interface, but it would also stop the process. |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sat Aug 10, 2013 8:09 am Post subject: |
|
|
| smartass wrote: | | Thank you for sharing this howto VinzC |
My pleasure .
| smartass wrote: | I'm using OpenRC with dhcpcd and ifplugd and noticed that it's almost redundant, because dhcpcd can do what ifplugd does and having several dhcpcd processes started by OpenRC means you get more default gateways and metrics may not be optimal.
However, a major issue for me with such an automagic dhcpcd setup is that I'm not sure how easy it would be to remove an interface from dhcpcd managed pool and configure it temporarily for some debugging, e.g. some static IP or MAC address.
Are you aware of a way to do this with this automagic dhcpcd exclusive setup?
The only way I could think of is making changes to dhcpcd.conf and then sending some signal to the dhcpcd process to re-read the config file.
However, I couldn't find any signal interface in man dhcpcd, but --rebind seems close, but it will temporarily disable ALL interfaces, I was hoping for something that would operate only on a specific interface and wouldn't interrupt others. --release would operate on the specified interface, but it would also stop the process. |
I'd have thought of using --release too indeed. Have you tried --release <interface> ?
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
smartass
Apprentice
Joined: 04 Jul 2011
Posts: 189
Location: right behind you ... (you did turn around, didn't you?)
|
| Posted: Sat Aug 10, 2013 11:13 am Post subject: |
|
|
--release can be applied to a specific interface, but according to the manpage it also causes the managing process to exit
| man dhcpcd wrote: |
-k, --release
This causes an existing dhcpcd process running on the interface
to release its lease, de-configure the interface and then exit.
dhcpcd then waits until this process has exited.
|
I'm not sure how dhcpcd is designed, but if it spawned a subprocess for each interface then this approach would be viable.
Does --release <interface> kill the main dhcpcd process (which would be managing at least 2 interfaces) on your setup VinzC? |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sat Aug 10, 2013 3:32 pm Post subject: |
|
|
| smartass wrote: | | Does --release <interface> kill the main dhcpcd process (which would be managing at least 2 interfaces) on your setup VinzC? |
 the documentation needs some adjustments indeed. I tried dhcpcd --release wlan0 on my laptop, which has eth0 and wlan0 setup with the same instance of dhcpcd; --release wlan0 didn't make the daemon process exit, eth0 still had its IP address. So you can safely run dhcpcd --release <interface> without interfering with the other interfaces; as well the dhcpcd daemon will not exit.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
smartass
Apprentice
Joined: 04 Jul 2011
Posts: 189
Location: right behind you ... (you did turn around, didn't you?)
|
| Posted: Sat Aug 10, 2013 3:43 pm Post subject: |
|
|
oh, that's nice
So if this works, how would I get it back to normal after I don't need the temporarily released interface anymore?
How did you do it? restarted the main dhcpcd process?
I'd like to do it without restarting the whole main process so I wouldn't interrupt any communication going through other interfaces, right now --rebind looks like the best option, but according to manpage
| man dhcpcd wrote: |
-n, --rebind
Notifies dhcpcd to reload its configuration and rebind its interfaces.
If dhcpcd is not running, then it starts up as normal.
|
Sounds like it would reconfigure all interfaces, effectively being the same problem like restarting the main process. |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sat Aug 10, 2013 7:57 pm Post subject: |
|
|
| smartass wrote: | | How did you do it? |
I ran dhcpcd --rebind and effectively all the interfaces got a new lease, the ones with an existing lease got it renewed.
| smartass wrote: | I'd like to do it without restarting the whole main process so I wouldn't interrupt any communication going through other interfaces,
|
I understand it could be a problem if you are "unbinding" more than one interface at the same time. Otherwise why would it be a problem?
| smartass wrote: | right now --rebind looks like the best option, but according to manpage
| man dhcpcd wrote: |
-n, --rebind
Notifies dhcpcd to reload its configuration and rebind its interfaces.
If dhcpcd is not running, then it starts up as normal.
|
Sounds like it would reconfigure all interfaces, effectively being the same problem like restarting the main process. |
Just do the same as I did: try dhcpcd --rebind <interface> then and see if it works .
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
smartass
Apprentice
Joined: 04 Jul 2011
Posts: 189
Location: right behind you ... (you did turn around, didn't you?)
|
| Posted: Sun Aug 11, 2013 2:50 am Post subject: |
|
|
I tested --rebind with and without an interface argument. Without an interface argument it behaves as in the man page, with one it rebinds only the specified interface. Yay! I'll send Roy Marples a patch for the manpage (I've already put up a ticket).
| VinzC wrote: |
I understand it could be a problem if you are "unbinding" more than one interface at the same time. Otherwise why would it be a problem?
|
If all interfaces got rebinded, an ongoing connection with some protocol that easily breaks could break because the connection would be temporarily interrupted.
A few points for your howto:
- you should rather use the default runlevel, that's what most people use. Simply add a statement that any runlevel can be used, but wpa_supplicant and dhcpcd should be in the same runlevel for the dependency resolution to work properly.
- I'd like to see this eventually on the wiki as an extension of my OpenRC roaming howto which is based on net.* scripts right now and your dhcpcd approach could be an alternative approach or possibly the main one, because it's more roaming-like, the original one could be left as an alternative approach for people that need the power of net.* scripts
|
|
| Back to top |
|
|
smartass
Apprentice
Joined: 04 Jul 2011
Posts: 189
Location: right behind you ... (you did turn around, didn't you?)
|
| Posted: Sun Aug 11, 2013 8:49 am Post subject: |
|
|
I've started using this setup to test it, I like how dhcpcd is automagic
However, there is one slight inconvenience coming from the fact that dhcpcd does not report state to OpenRC as with ifplugd (which can report "inactive"):
- When booting, OpenRC waits for dhcpcd to obtain a lease to provide the net virtual. When no network is available, this means I have to wait for the whole timeout before I can proceed booting.
- After dhcpcd is up and running and report as "started", if I loose my network connection, OpenRC will still think that net is available.
- After rfkilling wireless, wpa_supplicant doesn't seem to get back to manage it (I checked that it's not blocked with rfkill command), because that puts the interface down and wpa_supplicant cannot put it up itself (that's what the init script does).
How do you deal with these issues? |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sun Aug 11, 2013 10:57 am Post subject: |
|
|
@smartass:
| Quote: | | When booting, OpenRC waits for dhcpcd to obtain a lease to provide the net virtual. When no network is available, this means I have to wait for the whole timeout before I can proceed booting. |
I'm not too sure of that. At least what you describe doesn't happen on my laptop, which does not run wpa_supplicant by defaut and I often start my lappy without the cable plugged (not counting when I simply forget the cable). This means I often start my computer and there is no network available except the loopback interface. Besides dhcpcd backgrounds immediately to minimize service startup latency.
I never noticed any sort of time out and always saw my laptop boot in the same timely fashion as when OpenRC was managing the network. I'd have believed since the loopback adapter is always present and configured the network is always considered on. But that's just a guess.
| Quote: | | After dhcpcd is up and running and report as "started", if I loose my network connection, OpenRC will still think that net is available. |
See above. I also never considered it a problem as I always lost connections temporarily. If the computer starts being sluggish then it's the indication there's a network problem somewhere. I never experienced such a situation since I applied this solution though.
| Quote: | | After rfkilling wireless, wpa_supplicant doesn't seem to get back to manage it (I checked that it's not blocked with rfkill command), because that puts the interface down and wpa_supplicant cannot put it up itself (that's what the init script does). |
I simply never rfkill! I always start/stop wpa_supplicant when I need the wireless network. I always plug a cable whenever possible and only start wpa_supplicant otherwise. So in your case you might run | Code: | | /etc/init.d/wpa_supplicant stop |
instead of rfkilling. Also I (manually) unload the wireless card module when wireless is stopped. Never had an issue.
| Quote: | | you should rather use the default runlevel, that's what most people use. Simply add a statement that any runlevel can be used, but wpa_supplicant and dhcpcd should be in the same runlevel for the dependency resolution to work properly. |
Exact. FYI I used the boot runlevel to have the network start ASAP (as long as the cable is plugged beforehand, of course ).
| Quote: | | I'd like to see this eventually on the wiki as an extension of my OpenRC roaming howto which is based on net.* scripts right now and your dhcpcd approach could be an alternative approach or possibly the main one, because it's more roaming-like, the original one could be left as an alternative approach for people that need the power of net.* scripts |
I might do that .
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 547
Location: NRW, Germany
|
| Posted: Sun Aug 11, 2013 11:11 am Post subject: |
|
|
| smartass wrote: | | When booting, OpenRC waits for dhcpcd to obtain a lease to provide the net virtual. When no network is available, this means I have to wait for the whole timeout before I can proceed booting. |
--background
| smartass wrote: | | After dhcpcd is up and running and report as "started", if I loose my network connection, OpenRC will still think that net is available. |
no idea, sorry
| smartass wrote: | | After rfkilling wireless, wpa_supplicant doesn't seem to get back to manage it (I checked that it's not blocked with rfkill command), because that puts the interface down and wpa_supplicant cannot put it up itself (that's what the init script does). |
I don't know what rfkill does, but from a look at it's manpage I don't see much difference to wpa_cli.
What do you use rfkill for? |
|
| Back to top |
|
|
smartass
Apprentice
Joined: 04 Jul 2011
Posts: 189
Location: right behind you ... (you did turn around, didn't you?)
|
| Posted: Sun Aug 11, 2013 12:09 pm Post subject: |
|
|
using --background is the obvious solution, but the dhcpcd init script offers little choice to set it in clean way through /etc/conf.d/
when I don't want to use wireless, I press the wifi switch button on my laptop which is hardware rfkill. It's a lot more convenient than stopping a whole init script. |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sun Aug 11, 2013 12:53 pm Post subject: |
|
|
| smartass wrote: | | when I don't want to use wireless, I press the wifi switch button on my laptop which is hardware rfkill. It's a lot more convenient than stopping a whole init script. |
Ah, ok, I see now. On my laptop there's no separate control over Wifi and Bluetooth, both are control with the same key/switch; so I chose to assign Bluetooth control to the RF switch and leave wireless control manual.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
Dr.Willy
Guru
Joined: 15 Jul 2007
Posts: 547
Location: NRW, Germany
|
| Posted: Sun Aug 11, 2013 6:34 pm Post subject: |
|
|
| smartass wrote: | | using --background is the obvious solution, but the dhcpcd init script offers little choice to set it in clean way through /etc/conf.d/ |
man dhcpcd.conf |
|
| Back to top |
|
|
smartass
Apprentice
Joined: 04 Jul 2011
Posts: 189
Location: right behind you ... (you did turn around, didn't you?)
|
| Posted: Sun Aug 11, 2013 6:45 pm Post subject: |
|
|
Thanks Dr. Willy, I should RTFM indeed
I did know about dhcpcd.conf, but I wasn't aware it can set so many things.
VinzC, I use rfkill because AFAIK it can save power (may not be true for all cards). I recommend you assign a special key you don't use very often to issue rfkill through your WM or DE. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
